[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785988004637995-12279-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 03:46:44 dist-test-slave-6ntk krb5kdc[12288](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 03:46:44 dist-test-slave-6ntk krb5kdc[12288](info): set up 2 sockets
Aug 06 03:46:44 dist-test-slave-6ntk krb5kdc[12288](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260806 03:46:45.990084 12279 mini_kdc.cc:121] Time spent starting KDC: real 1.332s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785988004637995-12279-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785988004637995-12279-0/krb5kdc/test-user.keytab.
Aug 06 03:46:46 dist-test-slave-6ntk krb5kdc[12288](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988006, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T03:46:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T03:46:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.253.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.253.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.253.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.254.keytab.
Entry for principal kudu/127.11.253.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.254.keytab.
I20260806 03:46:46.124415 12279 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskToVWRe/build/release/bin/kudu
/tmp/dist-test-taskToVWRe/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785988004637995-12279-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785988004637995-12279-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785988004637995-12279-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785988004637995-12279-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.253.254:45087
--webserver_interface=127.11.253.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.254.keytab
--principal=kudu/127.11.253.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.253.212:46285
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.253.254:45087
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785988004637995-12279-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785988004637995-12279-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785988004637995-12279-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:46:46.216241 12305 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:46:46.216506 12305 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:46:46.216547 12305 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:46:46.217778 12305 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 03:46:46.217849 12305 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:46:46.217872 12305 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:46:46.217912 12305 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 03:46:46.217941 12305 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 03:46:46.219655 12305 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.253.212:46285
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785988004637995-12279-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785988004637995-12279-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.253.254:45087
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.254.keytab
--principal=kudu/127.11.253.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.253.254:45087
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785988004637995-12279-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.253.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12305
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785988004637995-12279-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:46:46.219971 12305 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:46:46.220227 12305 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:46:46.226060 12312 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:46:46.226049 12318 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:46:46.226051 12314 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:46:46.226079 12305 server_base.cc:1034] running on GCE node
I20260806 03:46:46.226581 12305 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:46:46.226985 12305 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:46:46.228165 12305 hybrid_clock.cc:648] HybridClock initialized: now 1785988006228154 us; error 58 us; skew 500 ppm
Aug 06 03:46:46 dist-test-slave-6ntk krb5kdc[12288](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988006, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.253.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:46:46.234136 12305 init.cc:377] Logged in from keytab as kudu/127.11.253.254@KRBTEST.COM (short username kudu)
I20260806 03:46:46.234745 12305 webserver.cc:460] Webserver started at http://127.11.253.254:40855/ using document root <none> and password file <none>
I20260806 03:46:46.235010 12305 fs_manager.cc:359] Metadata directory not provided
I20260806 03:46:46.235080 12305 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:46:46.235239 12305 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:46:46.236402 12305 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785988004637995-12279-0/minicluster-data/master-0/data/instance:
uuid: "8096bcae9cb4419f8c6d13121ce74425"
format_stamp: "Formatted at 2026-08-06 03:46:46 on dist-test-slave-6ntk"
server_key: "0da6241d09acf38ea4a176d0a52f17b2"
server_key_iv: "8260ff57e17271d9945431bf7672b193"
server_key_version: "encryptionkey@0"
I20260806 03:46:46.236795 12305 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785988004637995-12279-0/minicluster-data/master-0/wal/instance:
uuid: "8096bcae9cb4419f8c6d13121ce74425"
format_stamp: "Formatted at 2026-08-06 03:46:46 on dist-test-slave-6ntk"
server_key: "0da6241d09acf38ea4a176d0a52f17b2"
server_key_iv: "8260ff57e17271d9945431bf7672b193"
server_key_version: "encryptionkey@0"
I20260806 03:46:46.238267 12305 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 03:46:46.239063 12325 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:46:46.239280 12305 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 03:46:46.239411 12305 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785988004637995-12279-0/minicluster-data/master-0/data,/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785988004637995-12279-0/minicluster-data/master-0/wal
uuid: "8096bcae9cb4419f8c6d13121ce74425"
format_stamp: "Formatted at 2026-08-06 03:46:46 on dist-test-slave-6ntk"
server_key: "0da6241d09acf38ea4a176d0a52f17b2"
server_key_iv: "8260ff57e17271d9945431bf7672b193"
server_key_version: "encryptionkey@0"
I20260806 03:46:46.239490 12305 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785988004637995-12279-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785988004637995-12279-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785988004637995-12279-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:46:46.279402 12305 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:46:46.281852 12305 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:46:46.282065 12305 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:46:46.286675 12305 rpc_server.cc:307] RPC server started. Bound to: 127.11.253.254:45087
I20260806 03:46:46.286720 12394 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.253.254:45087 every 8 connection(s)
I20260806 03:46:46.287171 12305 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785988004637995-12279-0/minicluster-data/master-0/data/info.pb
I20260806 03:46:46.288216 12398 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:46:46.290704 12279 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskToVWRe/build/release/bin/kudu as pid 12305
I20260806 03:46:46.290791 12279 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785988004637995-12279-0/minicluster-data/master-0/wal/instance
I20260806 03:46:46.291059 12398 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8096bcae9cb4419f8c6d13121ce74425: Bootstrap starting.
I20260806 03:46:46.291488 12279 external_mini_cluster.cc:1428] Setting key 278c0e372386d9a48e8b5cfa8f053d98
I20260806 03:46:46.291783 12398 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8096bcae9cb4419f8c6d13121ce74425: Neither blocks nor log segments found. Creating new log.
I20260806 03:46:46.292105 12398 log.cc:824] T 00000000000000000000000000000000 P 8096bcae9cb4419f8c6d13121ce74425: Log is configured to *not* fsync() on all Append() calls
I20260806 03:46:46.292938 12398 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8096bcae9cb4419f8c6d13121ce74425: No bootstrap required, opened a new log
I20260806 03:46:46.294847 12398 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8096bcae9cb4419f8c6d13121ce74425 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8096bcae9cb4419f8c6d13121ce74425" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 45087 } }
I20260806 03:46:46.295024 12398 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8096bcae9cb4419f8c6d13121ce74425 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:46:46.295063 12398 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8096bcae9cb4419f8c6d13121ce74425 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8096bcae9cb4419f8c6d13121ce74425, State: Initialized, Role: FOLLOWER
I20260806 03:46:46.295184 12398 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8096bcae9cb4419f8c6d13121ce74425 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8096bcae9cb4419f8c6d13121ce74425" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 45087 } }
I20260806 03:46:46.295326 12398 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8096bcae9cb4419f8c6d13121ce74425 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 03:46:46.295352 12398 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8096bcae9cb4419f8c6d13121ce74425 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 03:46:46.295423 12398 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8096bcae9cb4419f8c6d13121ce74425 [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:46:46.296309 12398 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8096bcae9cb4419f8c6d13121ce74425 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8096bcae9cb4419f8c6d13121ce74425" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 45087 } }
I20260806 03:46:46.296402 12398 leader_election.cc:302] T 00000000000000000000000000000000 P 8096bcae9cb4419f8c6d13121ce74425 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8096bcae9cb4419f8c6d13121ce74425; no voters:
I20260806 03:46:46.296588 12398 leader_election.cc:288] T 00000000000000000000000000000000 P 8096bcae9cb4419f8c6d13121ce74425 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 03:46:46.296615 12407 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8096bcae9cb4419f8c6d13121ce74425 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 03:46:46.296730 12407 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8096bcae9cb4419f8c6d13121ce74425 [term 1 LEADER]: Becoming Leader. State: Replica: 8096bcae9cb4419f8c6d13121ce74425, State: Running, Role: LEADER
I20260806 03:46:46.296859 12398 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8096bcae9cb4419f8c6d13121ce74425 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 03:46:46.296816 12407 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8096bcae9cb4419f8c6d13121ce74425 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8096bcae9cb4419f8c6d13121ce74425" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 45087 } }
I20260806 03:46:46.298599 12408 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8096bcae9cb4419f8c6d13121ce74425 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8096bcae9cb4419f8c6d13121ce74425" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8096bcae9cb4419f8c6d13121ce74425" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 45087 } } }
I20260806 03:46:46.298560 12409 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8096bcae9cb4419f8c6d13121ce74425 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8096bcae9cb4419f8c6d13121ce74425. Latest consensus state: current_term: 1 leader_uuid: "8096bcae9cb4419f8c6d13121ce74425" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8096bcae9cb4419f8c6d13121ce74425" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 45087 } } }
I20260806 03:46:46.298925 12409 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8096bcae9cb4419f8c6d13121ce74425 [sys.catalog]: This master's current role is: LEADER
I20260806 03:46:46.299154 12408 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8096bcae9cb4419f8c6d13121ce74425 [sys.catalog]: This master's current role is: LEADER
W20260806 03:46:46.299454 12424 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 8096bcae9cb4419f8c6d13121ce74425: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 03:46:46.299521 12424 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 03:46:46.299593 12426 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 03:46:46.299757 12426 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 03:46:46.301939 12426 catalog_manager.cc:1347] Generated new cluster ID: 49820badceb54370808a300eca726611
I20260806 03:46:46.301993 12426 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 03:46:46 dist-test-slave-6ntk krb5kdc[12288](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988006, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.253.254@KRBTEST.COM
I20260806 03:46:46.309732 12403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:46:46.292489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37578 (local address 127.11.253.254:45087)
0806 03:46:46.292682 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0806 03:46:46.292692 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 03:46:46.292924 (+ 232us) server_negotiation.cc:407] Connection header received
0806 03:46:46.294802 (+ 1878us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:46:46.294810 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:46:46.294870 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:46:46.295024 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:46:46.296956 (+ 1932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:46:46.298493 (+ 1537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:46:46.300158 (+ 1665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:46:46.300629 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:46:46.300772 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:46:46.305262 (+ 4490us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:46:46.305274 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:46:46.305275 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:46:46.305287 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:46:46.307552 (+ 2265us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:46:46.308048 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:46:46.308059 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:46:46.308060 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:46:46.308187 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:46:46.308473 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:46:46.308474 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:46:46.308475 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:46:46.308792 (+ 317us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:46:46.308964 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0806 03:46:46.309228 (+ 264us) server_negotiation.cc:299] Negotiation successful
0806 03:46:46.309437 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":82,"threads_started":1}
I20260806 03:46:46.327653 12426 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 03:46:46.328470 12426 catalog_manager.cc:1504] Loading token signing keys...
I20260806 03:46:46.344553 12426 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8096bcae9cb4419f8c6d13121ce74425: Generated new TSK 0
I20260806 03:46:46.344786 12426 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.253.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.253.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.253.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.193.keytab.
Entry for principal kudu/127.11.253.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.193.keytab.
I20260806 03:46:46.381072 12279 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskToVWRe/build/release/bin/kudu
/tmp/dist-test-taskToVWRe/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785988004637995-12279-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785988004637995-12279-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785988004637995-12279-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785988004637995-12279-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.253.193:0
--local_ip_for_outbound_sockets=127.11.253.193
--webserver_interface=127.11.253.193
--webserver_port=0
--tserver_master_addrs=127.11.253.254:45087
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.193.keytab
--principal=kudu/127.11.253.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.253.212:46285
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785988004637995-12279-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785988004637995-12279-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785988004637995-12279-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:46:46.466235 12431 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:46:46.466436 12431 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:46:46.466460 12431 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:46:46.467599 12431 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:46:46.467641 12431 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:46:46.467694 12431 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.253.193
I20260806 03:46:46.469187 12431 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.253.212:46285
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785988004637995-12279-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785988004637995-12279-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.193.keytab
--principal=kudu/127.11.253.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.253.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785988004637995-12279-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.253.193
--webserver_port=0
--tserver_master_addrs=127.11.253.254:45087
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12431
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.253.193
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785988004637995-12279-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:46:46.469439 12431 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:46:46.469688 12431 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:46:46.475159 12441 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:46:46.475178 12438 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:46:46.475169 12436 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:46:46.475344 12431 server_base.cc:1034] running on GCE node
I20260806 03:46:46.475623 12431 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:46:46.475886 12431 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:46:46.477029 12431 hybrid_clock.cc:648] HybridClock initialized: now 1785988006477017 us; error 29 us; skew 500 ppm
Aug 06 03:46:46 dist-test-slave-6ntk krb5kdc[12288](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988006, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.253.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:46:46.479684 12431 init.cc:377] Logged in from keytab as kudu/127.11.253.193@KRBTEST.COM (short username kudu)
I20260806 03:46:46.480135 12431 webserver.cc:460] Webserver started at http://127.11.253.193:43347/ using document root <none> and password file <none>
I20260806 03:46:46.480320 12431 fs_manager.cc:359] Metadata directory not provided
I20260806 03:46:46.480362 12431 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:46:46.480449 12431 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:46:46.481688 12431 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785988004637995-12279-0/minicluster-data/ts-0/data/instance:
uuid: "0a68d0455c7042b985cac71dc0791ba6"
format_stamp: "Formatted at 2026-08-06 03:46:46 on dist-test-slave-6ntk"
server_key: "48d3dbc0f72730f907237913f87fc0ab"
server_key_iv: "9e472d89991b9d16b3b4829e8fd471ba"
server_key_version: "encryptionkey@0"
I20260806 03:46:46.482137 12431 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785988004637995-12279-0/minicluster-data/ts-0/wal/instance:
uuid: "0a68d0455c7042b985cac71dc0791ba6"
format_stamp: "Formatted at 2026-08-06 03:46:46 on dist-test-slave-6ntk"
server_key: "48d3dbc0f72730f907237913f87fc0ab"
server_key_iv: "9e472d89991b9d16b3b4829e8fd471ba"
server_key_version: "encryptionkey@0"
I20260806 03:46:46.483588 12431 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 03:46:46.484320 12451 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:46:46.484526 12431 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 03:46:46.484643 12431 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785988004637995-12279-0/minicluster-data/ts-0/data,/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785988004637995-12279-0/minicluster-data/ts-0/wal
uuid: "0a68d0455c7042b985cac71dc0791ba6"
format_stamp: "Formatted at 2026-08-06 03:46:46 on dist-test-slave-6ntk"
server_key: "48d3dbc0f72730f907237913f87fc0ab"
server_key_iv: "9e472d89991b9d16b3b4829e8fd471ba"
server_key_version: "encryptionkey@0"
I20260806 03:46:46.484728 12431 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785988004637995-12279-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785988004637995-12279-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785988004637995-12279-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:46:46.500877 12431 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:46:46.502903 12431 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:46:46.503077 12431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:46:46.503327 12431 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 03:46:46.503667 12431 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 03:46:46.503724 12431 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 03:46:46.503774 12431 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 03:46:46.503806 12431 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 03:46:46.510733 12431 rpc_server.cc:307] RPC server started. Bound to: 127.11.253.193:44125
I20260806 03:46:46.510782 12594 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.253.193:44125 every 8 connection(s)
I20260806 03:46:46.511170 12431 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785988004637995-12279-0/minicluster-data/ts-0/data/info.pb
I20260806 03:46:46.516907 12279 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskToVWRe/build/release/bin/kudu as pid 12431
I20260806 03:46:46.517038 12279 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785988004637995-12279-0/minicluster-data/ts-0/wal/instance
I20260806 03:46:46.517199 12279 external_mini_cluster.cc:1428] Setting key 62f9f1eadd0d1ad32d095339d255ea81
Aug 06 03:46:46 dist-test-slave-6ntk krb5kdc[12288](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988006, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.253.193@KRBTEST.COM for kudu/127.11.253.254@KRBTEST.COM
I20260806 03:46:46.522115 12403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:46:46.512012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.253.193:56595 (local address 127.11.253.254:45087)
0806 03:46:46.512103 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0806 03:46:46.512105 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:46:46.512317 (+ 212us) server_negotiation.cc:407] Connection header received
0806 03:46:46.512811 (+ 494us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:46:46.512813 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:46:46.512829 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:46:46.512886 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:46:46.514039 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:46:46.514563 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:46:46.515595 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:46:46.515823 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:46:46.515883 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:46:46.518551 (+ 2668us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:46:46.518552 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:46:46.518552 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:46:46.518559 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:46:46.520387 (+ 1828us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:46:46.520853 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:46:46.520855 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:46:46.520855 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:46:46.520930 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:46:46.521389 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:46:46.521390 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:46:46.521390 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:46:46.521505 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:46:46.521563 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0806 03:46:46.521970 (+ 407us) server_negotiation.cc:299] Negotiation successful
0806 03:46:46.522046 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260806 03:46:46.522436 12598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:46:46.511961 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:45087 (local address 127.11.253.193:56595)
0806 03:46:46.512160 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0806 03:46:46.512198 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0806 03:46:46.512721 (+ 523us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:46:46.512943 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:46:46.512946 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:46:46.513660 (+ 714us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:46:46.513935 (+ 275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:46:46.513941 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:46:46.514660 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:46:46.514661 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:46:46.515507 (+ 846us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:46:46.515510 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:46:46.515906 (+ 396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:46:46.515906 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:46:46.516008 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:46:46.516604 (+ 596us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:46:46.516613 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:46:46.518441 (+ 1828us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:46:46.520501 (+ 2060us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:46:46.520509 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:46:46.520510 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:46:46.520771 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:46:46.521005 (+ 234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:46:46.521006 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:46:46.521006 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:46:46.521293 (+ 287us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:46:46.521578 (+ 285us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:46:46.521585 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:46:46.521792 (+ 207us) client_negotiation.cc:769] Sending connection context
0806 03:46:46.521954 (+ 162us) client_negotiation.cc:240] Negotiation successful
0806 03:46:46.522147 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":114,"threads_started":1}
I20260806 03:46:46.523070 12596 heartbeater.cc:347] Connected to a master server at 127.11.253.254:45087
I20260806 03:46:46.523176 12596 heartbeater.cc:464] Registering TS with master...
I20260806 03:46:46.523393 12596 heartbeater.cc:511] Master 127.11.253.254:45087 requested a full tablet report, sending...
I20260806 03:46:46.523967 12345 ts_manager.cc:194] Registered new tserver with Master: 0a68d0455c7042b985cac71dc0791ba6 (127.11.253.193:44125)
I20260806 03:46:46.524659 12345 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.253.193@KRBTEST.COM'} at 127.11.253.193:56595
WARNING: no policy specified for kudu/127.11.253.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.253.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.253.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.194.keytab.
Entry for principal kudu/127.11.253.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.194.keytab.
I20260806 03:46:46.548127 12279 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskToVWRe/build/release/bin/kudu
/tmp/dist-test-taskToVWRe/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785988004637995-12279-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785988004637995-12279-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785988004637995-12279-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785988004637995-12279-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.253.194:0
--local_ip_for_outbound_sockets=127.11.253.194
--webserver_interface=127.11.253.194
--webserver_port=0
--tserver_master_addrs=127.11.253.254:45087
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.194.keytab
--principal=kudu/127.11.253.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.253.212:46285
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785988004637995-12279-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785988004637995-12279-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785988004637995-12279-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:46:46.634070 12604 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:46:46.634346 12604 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:46:46.634389 12604 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:46:46.635610 12604 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:46:46.635692 12604 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:46:46.635740 12604 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.253.194
I20260806 03:46:46.637307 12604 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.253.212:46285
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785988004637995-12279-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785988004637995-12279-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.194.keytab
--principal=kudu/127.11.253.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.253.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785988004637995-12279-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.253.194
--webserver_port=0
--tserver_master_addrs=127.11.253.254:45087
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12604
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.253.194
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785988004637995-12279-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:46:46.637624 12604 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:46:46.637923 12604 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:46:46.643319 12612 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:46:46.643520 12614 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:46:46.643484 12604 server_base.cc:1034] running on GCE node
W20260806 03:46:46.643455 12610 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:46:46.643827 12604 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:46:46.644083 12604 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:46:46.645256 12604 hybrid_clock.cc:648] HybridClock initialized: now 1785988006645242 us; error 29 us; skew 500 ppm
Aug 06 03:46:46 dist-test-slave-6ntk krb5kdc[12288](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988006, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.253.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:46:46.647886 12604 init.cc:377] Logged in from keytab as kudu/127.11.253.194@KRBTEST.COM (short username kudu)
I20260806 03:46:46.648412 12604 webserver.cc:460] Webserver started at http://127.11.253.194:34561/ using document root <none> and password file <none>
I20260806 03:46:46.648672 12604 fs_manager.cc:359] Metadata directory not provided
I20260806 03:46:46.648743 12604 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:46:46.648898 12604 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:46:46.650125 12604 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785988004637995-12279-0/minicluster-data/ts-1/data/instance:
uuid: "0a2dd56e250a4ae1836e107883f9daa3"
format_stamp: "Formatted at 2026-08-06 03:46:46 on dist-test-slave-6ntk"
server_key: "4fe2475794c6bf4403748635bd2ff29d"
server_key_iv: "167f9b8ab5e5c531967195aa78914f5b"
server_key_version: "encryptionkey@0"
I20260806 03:46:46.650537 12604 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785988004637995-12279-0/minicluster-data/ts-1/wal/instance:
uuid: "0a2dd56e250a4ae1836e107883f9daa3"
format_stamp: "Formatted at 2026-08-06 03:46:46 on dist-test-slave-6ntk"
server_key: "4fe2475794c6bf4403748635bd2ff29d"
server_key_iv: "167f9b8ab5e5c531967195aa78914f5b"
server_key_version: "encryptionkey@0"
I20260806 03:46:46.651935 12604 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 03:46:46.652674 12623 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:46:46.652853 12604 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 03:46:46.652966 12604 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785988004637995-12279-0/minicluster-data/ts-1/data,/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785988004637995-12279-0/minicluster-data/ts-1/wal
uuid: "0a2dd56e250a4ae1836e107883f9daa3"
format_stamp: "Formatted at 2026-08-06 03:46:46 on dist-test-slave-6ntk"
server_key: "4fe2475794c6bf4403748635bd2ff29d"
server_key_iv: "167f9b8ab5e5c531967195aa78914f5b"
server_key_version: "encryptionkey@0"
I20260806 03:46:46.653039 12604 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785988004637995-12279-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785988004637995-12279-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785988004637995-12279-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:46:46.678017 12604 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:46:46.680516 12604 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:46:46.680683 12604 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:46:46.680943 12604 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 03:46:46.681272 12604 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 03:46:46.681329 12604 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 03:46:46.681402 12604 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 03:46:46.681434 12604 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 03:46:46.689826 12604 rpc_server.cc:307] RPC server started. Bound to: 127.11.253.194:38609
I20260806 03:46:46.689924 12772 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.253.194:38609 every 8 connection(s)
I20260806 03:46:46.690254 12604 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785988004637995-12279-0/minicluster-data/ts-1/data/info.pb
Aug 06 03:46:46 dist-test-slave-6ntk krb5kdc[12288](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988006, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.253.194@KRBTEST.COM for kudu/127.11.253.254@KRBTEST.COM
I20260806 03:46:46.696964 12279 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskToVWRe/build/release/bin/kudu as pid 12604
I20260806 03:46:46.697059 12279 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785988004637995-12279-0/minicluster-data/ts-1/wal/instance
I20260806 03:46:46.697311 12279 external_mini_cluster.cc:1428] Setting key 65c86d7dbeec956e295eac1f9705d8b7
I20260806 03:46:46.700330 12403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:46:46.690961 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.253.194:50313 (local address 127.11.253.254:45087)
0806 03:46:46.691008 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0806 03:46:46.691010 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:46:46.691195 (+ 185us) server_negotiation.cc:407] Connection header received
0806 03:46:46.691693 (+ 498us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:46:46.691694 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:46:46.691704 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:46:46.691764 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:46:46.692709 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:46:46.693203 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:46:46.694241 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:46:46.694445 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:46:46.694501 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:46:46.696553 (+ 2052us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:46:46.696554 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:46:46.696554 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:46:46.696561 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:46:46.698505 (+ 1944us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:46:46.699343 (+ 838us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:46:46.699345 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:46:46.699345 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:46:46.699413 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:46:46.699643 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:46:46.699644 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:46:46.699644 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:46:46.699798 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:46:46.699870 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0806 03:46:46.700182 (+ 312us) server_negotiation.cc:299] Negotiation successful
0806 03:46:46.700256 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260806 03:46:46.700663 12777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:46:46.690903 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:45087 (local address 127.11.253.194:50313)
0806 03:46:46.691071 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0806 03:46:46.691115 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0806 03:46:46.691604 (+ 489us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:46:46.691862 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:46:46.691866 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:46:46.692435 (+ 569us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:46:46.692631 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:46:46.692636 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:46:46.693284 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:46:46.693285 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:46:46.694171 (+ 886us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:46:46.694173 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:46:46.694520 (+ 347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:46:46.694520 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:46:46.694622 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:46:46.695078 (+ 456us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:46:46.695086 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:46:46.696468 (+ 1382us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:46:46.698602 (+ 2134us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:46:46.698610 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:46:46.698613 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:46:46.699240 (+ 627us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:46:46.699479 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:46:46.699480 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:46:46.699480 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:46:46.699579 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:46:46.699869 (+ 290us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:46:46.699873 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:46:46.700042 (+ 169us) client_negotiation.cc:769] Sending connection context
0806 03:46:46.700157 (+ 115us) client_negotiation.cc:240] Negotiation successful
0806 03:46:46.700331 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":94,"threads_started":1}
I20260806 03:46:46.701145 12774 heartbeater.cc:347] Connected to a master server at 127.11.253.254:45087
I20260806 03:46:46.701226 12774 heartbeater.cc:464] Registering TS with master...
I20260806 03:46:46.701440 12774 heartbeater.cc:511] Master 127.11.253.254:45087 requested a full tablet report, sending...
I20260806 03:46:46.701823 12345 ts_manager.cc:194] Registered new tserver with Master: 0a2dd56e250a4ae1836e107883f9daa3 (127.11.253.194:38609)
I20260806 03:46:46.702220 12345 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.253.194@KRBTEST.COM'} at 127.11.253.194:50313
WARNING: no policy specified for kudu/127.11.253.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.253.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.253.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.195.keytab.
Entry for principal kudu/127.11.253.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.195.keytab.
I20260806 03:46:46.726759 12279 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskToVWRe/build/release/bin/kudu
/tmp/dist-test-taskToVWRe/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785988004637995-12279-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785988004637995-12279-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785988004637995-12279-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785988004637995-12279-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.253.195:0
--local_ip_for_outbound_sockets=127.11.253.195
--webserver_interface=127.11.253.195
--webserver_port=0
--tserver_master_addrs=127.11.253.254:45087
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.195.keytab
--principal=kudu/127.11.253.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.253.212:46285
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785988004637995-12279-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785988004637995-12279-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785988004637995-12279-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:46:46.813059 12781 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:46:46.813294 12781 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:46:46.813335 12781 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:46:46.814558 12781 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:46:46.814623 12781 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:46:46.814669 12781 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.253.195
I20260806 03:46:46.816121 12781 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.253.212:46285
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785988004637995-12279-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785988004637995-12279-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.195.keytab
--principal=kudu/127.11.253.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.253.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785988004637995-12279-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.253.195
--webserver_port=0
--tserver_master_addrs=127.11.253.254:45087
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12781
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.253.195
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785988004637995-12279-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:46:46.816392 12781 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:46:46.816633 12781 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:46:46.822109 12791 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:46:46.822122 12787 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:46:46.822156 12788 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:46:46.822432 12781 server_base.cc:1034] running on GCE node
I20260806 03:46:46.822635 12781 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:46:46.822830 12781 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:46:46.823977 12781 hybrid_clock.cc:648] HybridClock initialized: now 1785988006823965 us; error 29 us; skew 500 ppm
Aug 06 03:46:46 dist-test-slave-6ntk krb5kdc[12288](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988006, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.253.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:46:46.826578 12781 init.cc:377] Logged in from keytab as kudu/127.11.253.195@KRBTEST.COM (short username kudu)
I20260806 03:46:46.827054 12781 webserver.cc:460] Webserver started at http://127.11.253.195:45733/ using document root <none> and password file <none>
I20260806 03:46:46.827241 12781 fs_manager.cc:359] Metadata directory not provided
I20260806 03:46:46.827281 12781 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:46:46.827373 12781 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:46:46.828377 12781 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785988004637995-12279-0/minicluster-data/ts-2/data/instance:
uuid: "3e70d370a1cf406fbe675ba0242f1ade"
format_stamp: "Formatted at 2026-08-06 03:46:46 on dist-test-slave-6ntk"
server_key: "b9088b55cdcf9148fc19105cf19a79de"
server_key_iv: "718f9242e85d482847314d1fda693781"
server_key_version: "encryptionkey@0"
I20260806 03:46:46.828696 12781 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785988004637995-12279-0/minicluster-data/ts-2/wal/instance:
uuid: "3e70d370a1cf406fbe675ba0242f1ade"
format_stamp: "Formatted at 2026-08-06 03:46:46 on dist-test-slave-6ntk"
server_key: "b9088b55cdcf9148fc19105cf19a79de"
server_key_iv: "718f9242e85d482847314d1fda693781"
server_key_version: "encryptionkey@0"
I20260806 03:46:46.830252 12781 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 03:46:46.830957 12801 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:46:46.831130 12781 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 03:46:46.831240 12781 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785988004637995-12279-0/minicluster-data/ts-2/data,/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785988004637995-12279-0/minicluster-data/ts-2/wal
uuid: "3e70d370a1cf406fbe675ba0242f1ade"
format_stamp: "Formatted at 2026-08-06 03:46:46 on dist-test-slave-6ntk"
server_key: "b9088b55cdcf9148fc19105cf19a79de"
server_key_iv: "718f9242e85d482847314d1fda693781"
server_key_version: "encryptionkey@0"
I20260806 03:46:46.831319 12781 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785988004637995-12279-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785988004637995-12279-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785988004637995-12279-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:46:46.858413 12781 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:46:46.860409 12781 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:46:46.860592 12781 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:46:46.860865 12781 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 03:46:46.861191 12781 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 03:46:46.861245 12781 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 03:46:46.861289 12781 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 03:46:46.861313 12781 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 03:46:46.869393 12781 rpc_server.cc:307] RPC server started. Bound to: 127.11.253.195:45559
I20260806 03:46:46.869443 12956 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.253.195:45559 every 8 connection(s)
I20260806 03:46:46.869827 12781 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785988004637995-12279-0/minicluster-data/ts-2/data/info.pb
I20260806 03:46:46.872888 12279 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskToVWRe/build/release/bin/kudu as pid 12781
I20260806 03:46:46.872997 12279 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785988004637995-12279-0/minicluster-data/ts-2/wal/instance
I20260806 03:46:46.873152 12279 external_mini_cluster.cc:1428] Setting key 9322a17fe7e5bb62d6333a76dbb053f4
Aug 06 03:46:46 dist-test-slave-6ntk krb5kdc[12288](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988006, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.253.195@KRBTEST.COM for kudu/127.11.253.254@KRBTEST.COM
I20260806 03:46:46.879772 12403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:46:46.870773 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.253.195:58375 (local address 127.11.253.254:45087)
0806 03:46:46.870874 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0806 03:46:46.870876 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:46:46.871069 (+ 193us) server_negotiation.cc:407] Connection header received
0806 03:46:46.871575 (+ 506us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:46:46.871577 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:46:46.871597 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:46:46.871660 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:46:46.872663 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:46:46.873336 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:46:46.874557 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:46:46.874786 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:46:46.874881 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:46:46.877117 (+ 2236us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:46:46.877118 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:46:46.877118 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:46:46.877125 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:46:46.878603 (+ 1478us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:46:46.878987 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:46:46.878988 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:46:46.878989 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:46:46.879040 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:46:46.879254 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:46:46.879255 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:46:46.879255 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:46:46.879358 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:46:46.879409 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0806 03:46:46.879644 (+ 235us) server_negotiation.cc:299] Negotiation successful
0806 03:46:46.879707 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260806 03:46:46.880055 12959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:46:46.870723 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:45087 (local address 127.11.253.195:58375)
0806 03:46:46.870940 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0806 03:46:46.870979 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0806 03:46:46.871465 (+ 486us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:46:46.871735 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:46:46.871739 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:46:46.872339 (+ 600us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:46:46.872548 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:46:46.872553 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:46:46.873439 (+ 886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:46:46.873440 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:46:46.874452 (+ 1012us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:46:46.874455 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:46:46.874891 (+ 436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:46:46.874892 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:46:46.875056 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:46:46.875543 (+ 487us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:46:46.875552 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:46:46.877030 (+ 1478us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:46:46.878673 (+ 1643us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:46:46.878680 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:46:46.878681 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:46:46.878920 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:46:46.879101 (+ 181us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:46:46.879102 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:46:46.879102 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:46:46.879201 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:46:46.879414 (+ 213us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:46:46.879418 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:46:46.879532 (+ 114us) client_negotiation.cc:769] Sending connection context
0806 03:46:46.879616 (+ 84us) client_negotiation.cc:240] Negotiation successful
0806 03:46:46.879736 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":136,"threads_started":1}
I20260806 03:46:46.880630 12957 heartbeater.cc:347] Connected to a master server at 127.11.253.254:45087
I20260806 03:46:46.880754 12957 heartbeater.cc:464] Registering TS with master...
I20260806 03:46:46.880973 12957 heartbeater.cc:511] Master 127.11.253.254:45087 requested a full tablet report, sending...
I20260806 03:46:46.881374 12345 ts_manager.cc:194] Registered new tserver with Master: 3e70d370a1cf406fbe675ba0242f1ade (127.11.253.195:45559)
I20260806 03:46:46.881747 12345 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.253.195@KRBTEST.COM'} at 127.11.253.195:58375
I20260806 03:46:46.885886 12279 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 06 03:46:46 dist-test-slave-6ntk krb5kdc[12288](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988006, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 06 03:46:46 dist-test-slave-6ntk krb5kdc[12288](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988006, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.11.253.193@KRBTEST.COM
I20260806 03:46:46.914716 12964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:46:46.906770 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35094 (local address 127.11.253.193:44125)
0806 03:46:46.906974 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0806 03:46:46.906979 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 03:46:46.906992 (+ 13us) server_negotiation.cc:407] Connection header received
0806 03:46:46.907067 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:46:46.907069 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:46:46.907088 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:46:46.907171 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:46:46.907782 (+ 611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:46:46.908501 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:46:46.909205 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:46:46.909583 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:46:46.909644 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:46:46.911880 (+ 2236us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:46:46.911889 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:46:46.911889 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:46:46.911896 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:46:46.913593 (+ 1697us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:46:46.913951 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:46:46.913954 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:46:46.913954 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:46:46.914013 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:46:46.914197 (+ 184us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:46:46.914198 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:46:46.914198 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:46:46.914336 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:46:46.914442 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0806 03:46:46.914515 (+ 73us) server_negotiation.cc:299] Negotiation successful
0806 03:46:46.914579 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":161,"thread_start_us":126,"threads_started":1}
W20260806 03:46:46.915143 12484 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35094
Aug 06 03:46:46 dist-test-slave-6ntk krb5kdc[12288](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988006, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 03:46:46 dist-test-slave-6ntk krb5kdc[12288](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988006, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.253.193@KRBTEST.COM
I20260806 03:46:46.942344 12964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:46:46.935037 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35106 (local address 127.11.253.193:44125)
0806 03:46:46.935125 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0806 03:46:46.935127 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:46:46.935227 (+ 100us) server_negotiation.cc:407] Connection header received
0806 03:46:46.935296 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:46:46.935297 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:46:46.935321 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:46:46.935381 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:46:46.935955 (+ 574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:46:46.936467 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:46:46.937178 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:46:46.937390 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:46:46.937442 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:46:46.939815 (+ 2373us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:46:46.939817 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:46:46.939817 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:46:46.939824 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:46:46.941258 (+ 1434us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:46:46.941559 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:46:46.941560 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:46:46.941560 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:46:46.941610 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:46:46.941757 (+ 147us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:46:46.941757 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:46:46.941757 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:46:46.941935 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:46:46.942060 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0806 03:46:46.942207 (+ 147us) server_negotiation.cc:299] Negotiation successful
0806 03:46:46.942276 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260806 03:46:46.942961 12279 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskToVWRe/build/release/bin/kudu with pid 12431
I20260806 03:46:46.950628 12279 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskToVWRe/build/release/bin/kudu with pid 12604
I20260806 03:46:46.962692 12279 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskToVWRe/build/release/bin/kudu with pid 12781
I20260806 03:46:46.971601 12279 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskToVWRe/build/release/bin/kudu with pid 12305
2026-08-06T03:46:46Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (2344 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785988004637995-12279-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 03:46:47 dist-test-slave-6ntk krb5kdc[12972](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 03:46:47 dist-test-slave-6ntk krb5kdc[12972](info): set up 2 sockets
Aug 06 03:46:47 dist-test-slave-6ntk krb5kdc[12972](info): commencing operation
krb5kdc: starting...
W20260806 03:46:48.187800 12279 mini_kdc.cc:121] Time spent starting KDC: real 1.190s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785988004637995-12279-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785988004637995-12279-0/krb5kdc/test-user.keytab.
Aug 06 03:46:48 dist-test-slave-6ntk krb5kdc[12972](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988008, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T03:46:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T03:46:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.253.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.253.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.253.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.254.keytab.
Entry for principal kudu/127.11.253.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.254.keytab.
I20260806 03:46:48.321277 12279 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskToVWRe/build/release/bin/kudu
/tmp/dist-test-taskToVWRe/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785988004637995-12279-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785988004637995-12279-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785988004637995-12279-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785988004637995-12279-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.253.254:45631
--webserver_interface=127.11.253.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.254.keytab
--principal=kudu/127.11.253.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.253.212:44585
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.253.254:45631
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785988004637995-12279-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785988004637995-12279-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785988004637995-12279-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:46:48.404623 12989 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:46:48.404911 12989 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:46:48.404950 12989 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:46:48.406267 12989 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 03:46:48.406351 12989 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:46:48.406378 12989 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:46:48.406407 12989 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 03:46:48.406438 12989 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 03:46:48.408120 12989 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.253.212:44585
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785988004637995-12279-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785988004637995-12279-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.253.254:45631
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.254.keytab
--principal=kudu/127.11.253.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.253.254:45631
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785988004637995-12279-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.253.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12989
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785988004637995-12279-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:46:48.408422 12989 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:46:48.408663 12989 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:46:48.413767 12996 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:46:48.413750 12995 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:46:48.413789 12998 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:46:48.414067 12989 server_base.cc:1034] running on GCE node
I20260806 03:46:48.414451 12989 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:46:48.414762 12989 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:46:48.415948 12989 hybrid_clock.cc:648] HybridClock initialized: now 1785988008415931 us; error 48 us; skew 500 ppm
Aug 06 03:46:48 dist-test-slave-6ntk krb5kdc[12972](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988008, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.253.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:46:48.418591 12989 init.cc:377] Logged in from keytab as kudu/127.11.253.254@KRBTEST.COM (short username kudu)
I20260806 03:46:48.419087 12989 webserver.cc:460] Webserver started at http://127.11.253.254:45893/ using document root <none> and password file <none>
I20260806 03:46:48.419361 12989 fs_manager.cc:359] Metadata directory not provided
I20260806 03:46:48.419435 12989 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:46:48.419590 12989 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:46:48.420739 12989 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785988004637995-12279-0/minicluster-data/master-0/data/instance:
uuid: "0198dc186e0e41c38436761409b4a611"
format_stamp: "Formatted at 2026-08-06 03:46:48 on dist-test-slave-6ntk"
server_key: "d42ef6934c58259ae5f502564656f475"
server_key_iv: "17f29f1b54c650ed1872233f596d35ee"
server_key_version: "encryptionkey@0"
I20260806 03:46:48.421115 12989 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785988004637995-12279-0/minicluster-data/master-0/wal/instance:
uuid: "0198dc186e0e41c38436761409b4a611"
format_stamp: "Formatted at 2026-08-06 03:46:48 on dist-test-slave-6ntk"
server_key: "d42ef6934c58259ae5f502564656f475"
server_key_iv: "17f29f1b54c650ed1872233f596d35ee"
server_key_version: "encryptionkey@0"
I20260806 03:46:48.422549 12989 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 03:46:48.423388 13013 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:46:48.423607 12989 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 03:46:48.423727 12989 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785988004637995-12279-0/minicluster-data/master-0/data,/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785988004637995-12279-0/minicluster-data/master-0/wal
uuid: "0198dc186e0e41c38436761409b4a611"
format_stamp: "Formatted at 2026-08-06 03:46:48 on dist-test-slave-6ntk"
server_key: "d42ef6934c58259ae5f502564656f475"
server_key_iv: "17f29f1b54c650ed1872233f596d35ee"
server_key_version: "encryptionkey@0"
I20260806 03:46:48.423806 12989 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785988004637995-12279-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785988004637995-12279-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785988004637995-12279-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:46:48.439678 12989 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:46:48.441741 12989 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:46:48.442001 12989 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:46:48.446447 12989 rpc_server.cc:307] RPC server started. Bound to: 127.11.253.254:45631
I20260806 03:46:48.446503 13089 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.253.254:45631 every 8 connection(s)
I20260806 03:46:48.446905 12989 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785988004637995-12279-0/minicluster-data/master-0/data/info.pb
I20260806 03:46:48.447890 13090 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:46:48.450328 13090 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0198dc186e0e41c38436761409b4a611: Bootstrap starting.
I20260806 03:46:48.450960 13090 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0198dc186e0e41c38436761409b4a611: Neither blocks nor log segments found. Creating new log.
I20260806 03:46:48.451268 13090 log.cc:824] T 00000000000000000000000000000000 P 0198dc186e0e41c38436761409b4a611: Log is configured to *not* fsync() on all Append() calls
I20260806 03:46:48.452028 13090 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0198dc186e0e41c38436761409b4a611: No bootstrap required, opened a new log
I20260806 03:46:48.453701 13090 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0198dc186e0e41c38436761409b4a611 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0198dc186e0e41c38436761409b4a611" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 45631 } }
I20260806 03:46:48.453852 13090 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0198dc186e0e41c38436761409b4a611 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:46:48.453948 13090 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0198dc186e0e41c38436761409b4a611 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0198dc186e0e41c38436761409b4a611, State: Initialized, Role: FOLLOWER
I20260806 03:46:48.454109 13090 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0198dc186e0e41c38436761409b4a611 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0198dc186e0e41c38436761409b4a611" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 45631 } }
I20260806 03:46:48.454177 13090 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0198dc186e0e41c38436761409b4a611 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 03:46:48.454200 13090 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0198dc186e0e41c38436761409b4a611 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 03:46:48.454268 13090 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0198dc186e0e41c38436761409b4a611 [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:46:48.454802 13090 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0198dc186e0e41c38436761409b4a611 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0198dc186e0e41c38436761409b4a611" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 45631 } }
I20260806 03:46:48.454926 13090 leader_election.cc:302] T 00000000000000000000000000000000 P 0198dc186e0e41c38436761409b4a611 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0198dc186e0e41c38436761409b4a611; no voters:
I20260806 03:46:48.455143 13090 leader_election.cc:288] T 00000000000000000000000000000000 P 0198dc186e0e41c38436761409b4a611 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 03:46:48.455159 13098 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0198dc186e0e41c38436761409b4a611 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 03:46:48.455379 13098 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0198dc186e0e41c38436761409b4a611 [term 1 LEADER]: Becoming Leader. State: Replica: 0198dc186e0e41c38436761409b4a611, State: Running, Role: LEADER
I20260806 03:46:48.455446 13090 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0198dc186e0e41c38436761409b4a611 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 03:46:48.455502 13098 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0198dc186e0e41c38436761409b4a611 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0198dc186e0e41c38436761409b4a611" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 45631 } }
I20260806 03:46:48.455885 13100 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0198dc186e0e41c38436761409b4a611 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0198dc186e0e41c38436761409b4a611. Latest consensus state: current_term: 1 leader_uuid: "0198dc186e0e41c38436761409b4a611" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0198dc186e0e41c38436761409b4a611" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 45631 } } }
I20260806 03:46:48.455974 13100 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0198dc186e0e41c38436761409b4a611 [sys.catalog]: This master's current role is: LEADER
I20260806 03:46:48.456059 13099 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0198dc186e0e41c38436761409b4a611 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0198dc186e0e41c38436761409b4a611" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0198dc186e0e41c38436761409b4a611" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 45631 } } }
I20260806 03:46:48.456135 13099 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0198dc186e0e41c38436761409b4a611 [sys.catalog]: This master's current role is: LEADER
I20260806 03:46:48.456218 13109 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 03:46:48.456655 12279 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskToVWRe/build/release/bin/kudu as pid 12989
I20260806 03:46:48.456748 12279 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785988004637995-12279-0/minicluster-data/master-0/wal/instance
I20260806 03:46:48.456858 13109 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 03:46:48.456934 12279 external_mini_cluster.cc:1428] Setting key fe04dcb966720fb0cfdf287c6c7cde5f
I20260806 03:46:48.462380 13109 catalog_manager.cc:1347] Generated new cluster ID: dbb1b8ee0d474795a405b6c4e61de5df
I20260806 03:46:48.462438 13109 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 03:46:48 dist-test-slave-6ntk krb5kdc[12972](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988008, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.253.254@KRBTEST.COM
I20260806 03:46:48.467618 13124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:46:48.457717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60024 (local address 127.11.253.254:45631)
0806 03:46:48.457940 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0806 03:46:48.457950 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 03:46:48.457984 (+ 34us) server_negotiation.cc:407] Connection header received
0806 03:46:48.458100 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:46:48.458107 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:46:48.458161 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:46:48.458473 (+ 312us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:46:48.459114 (+ 641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:46:48.460023 (+ 909us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:46:48.460783 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:46:48.461242 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:46:48.461377 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:46:48.464077 (+ 2700us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:46:48.464081 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:46:48.464081 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:46:48.464094 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:46:48.465956 (+ 1862us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:46:48.466336 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:46:48.466339 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:46:48.466339 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:46:48.466453 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:46:48.466677 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:46:48.466678 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:46:48.466678 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:46:48.466926 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:46:48.467114 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0806 03:46:48.467194 (+ 80us) server_negotiation.cc:299] Negotiation successful
0806 03:46:48.467444 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":81,"threads_started":1}
I20260806 03:46:48.469477 13109 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 03:46:48.470072 13109 catalog_manager.cc:1504] Loading token signing keys...
I20260806 03:46:48.475553 13109 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0198dc186e0e41c38436761409b4a611: Generated new TSK 0
I20260806 03:46:48.475762 13109 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.253.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.253.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.253.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.193.keytab.
Entry for principal kudu/127.11.253.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.193.keytab.
I20260806 03:46:48.513192 12279 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskToVWRe/build/release/bin/kudu
/tmp/dist-test-taskToVWRe/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785988004637995-12279-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785988004637995-12279-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785988004637995-12279-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785988004637995-12279-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.253.193:0
--local_ip_for_outbound_sockets=127.11.253.193
--webserver_interface=127.11.253.193
--webserver_port=0
--tserver_master_addrs=127.11.253.254:45631
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.193.keytab
--principal=kudu/127.11.253.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.253.212:44585
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785988004637995-12279-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785988004637995-12279-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785988004637995-12279-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:46:48.607620 13131 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:46:48.607836 13131 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:46:48.607860 13131 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:46:48.609031 13131 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:46:48.609105 13131 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:46:48.609155 13131 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.253.193
I20260806 03:46:48.610734 13131 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.253.212:44585
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785988004637995-12279-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785988004637995-12279-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.193.keytab
--principal=kudu/127.11.253.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.253.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785988004637995-12279-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.253.193
--webserver_port=0
--tserver_master_addrs=127.11.253.254:45631
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13131
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.253.193
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785988004637995-12279-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:46:48.611020 13131 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:46:48.611270 13131 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:46:48.616647 13142 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:46:48.616655 13139 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:46:48.616664 13144 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:46:48.617033 13131 server_base.cc:1034] running on GCE node
I20260806 03:46:48.617264 13131 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:46:48.617480 13131 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:46:48.618633 13131 hybrid_clock.cc:648] HybridClock initialized: now 1785988008618620 us; error 28 us; skew 500 ppm
Aug 06 03:46:48 dist-test-slave-6ntk krb5kdc[12972](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988008, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.253.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:46:48.621335 13131 init.cc:377] Logged in from keytab as kudu/127.11.253.193@KRBTEST.COM (short username kudu)
I20260806 03:46:48.621961 13131 webserver.cc:460] Webserver started at http://127.11.253.193:41135/ using document root <none> and password file <none>
I20260806 03:46:48.622227 13131 fs_manager.cc:359] Metadata directory not provided
I20260806 03:46:48.622299 13131 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:46:48.622452 13131 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:46:48.623512 13131 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785988004637995-12279-0/minicluster-data/ts-0/data/instance:
uuid: "14542d79c36443f08164ae20dd0762fd"
format_stamp: "Formatted at 2026-08-06 03:46:48 on dist-test-slave-6ntk"
server_key: "2cb4d2d3baa1d85613fc889efe11ffbf"
server_key_iv: "4406cb42c7de53bc9806cbdcf09bdeac"
server_key_version: "encryptionkey@0"
I20260806 03:46:48.623881 13131 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785988004637995-12279-0/minicluster-data/ts-0/wal/instance:
uuid: "14542d79c36443f08164ae20dd0762fd"
format_stamp: "Formatted at 2026-08-06 03:46:48 on dist-test-slave-6ntk"
server_key: "2cb4d2d3baa1d85613fc889efe11ffbf"
server_key_iv: "4406cb42c7de53bc9806cbdcf09bdeac"
server_key_version: "encryptionkey@0"
I20260806 03:46:48.625370 13131 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 03:46:48.626219 13153 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:46:48.626482 13131 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 03:46:48.626616 13131 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785988004637995-12279-0/minicluster-data/ts-0/data,/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785988004637995-12279-0/minicluster-data/ts-0/wal
uuid: "14542d79c36443f08164ae20dd0762fd"
format_stamp: "Formatted at 2026-08-06 03:46:48 on dist-test-slave-6ntk"
server_key: "2cb4d2d3baa1d85613fc889efe11ffbf"
server_key_iv: "4406cb42c7de53bc9806cbdcf09bdeac"
server_key_version: "encryptionkey@0"
I20260806 03:46:48.626699 13131 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785988004637995-12279-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785988004637995-12279-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785988004637995-12279-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:46:48.643620 13131 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:46:48.645694 13131 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:46:48.645959 13131 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:46:48.646230 13131 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 03:46:48.646570 13131 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 03:46:48.646628 13131 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 03:46:48.646674 13131 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 03:46:48.646699 13131 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 03:46:48.652835 13131 rpc_server.cc:307] RPC server started. Bound to: 127.11.253.193:44463
I20260806 03:46:48.652877 13308 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.253.193:44463 every 8 connection(s)
I20260806 03:46:48.653268 13131 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785988004637995-12279-0/minicluster-data/ts-0/data/info.pb
I20260806 03:46:48.659996 12279 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskToVWRe/build/release/bin/kudu as pid 13131
I20260806 03:46:48.660080 12279 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785988004637995-12279-0/minicluster-data/ts-0/wal/instance
Aug 06 03:46:48 dist-test-slave-6ntk krb5kdc[12972](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988008, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.253.193@KRBTEST.COM for kudu/127.11.253.254@KRBTEST.COM
I20260806 03:46:48.660302 12279 external_mini_cluster.cc:1428] Setting key 069ef8f9908bf27c39d6a2b4d43bd595
I20260806 03:46:48.663777 13124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:46:48.654149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.253.193:34747 (local address 127.11.253.254:45631)
0806 03:46:48.654199 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0806 03:46:48.654201 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:46:48.654437 (+ 236us) server_negotiation.cc:407] Connection header received
0806 03:46:48.655583 (+ 1146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:46:48.655585 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:46:48.655602 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:46:48.655653 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:46:48.656347 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:46:48.656929 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:46:48.657992 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:46:48.658222 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:46:48.658317 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:46:48.660746 (+ 2429us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:46:48.660747 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:46:48.660748 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:46:48.660756 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:46:48.662317 (+ 1561us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:46:48.662762 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:46:48.662764 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:46:48.662765 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:46:48.662825 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:46:48.663103 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:46:48.663104 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:46:48.663104 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:46:48.663221 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:46:48.663300 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0806 03:46:48.663590 (+ 290us) server_negotiation.cc:299] Negotiation successful
0806 03:46:48.663687 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 03:46:48.664011 13311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:46:48.654129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:45631 (local address 127.11.253.193:34747)
0806 03:46:48.654312 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0806 03:46:48.654351 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0806 03:46:48.655458 (+ 1107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:46:48.655755 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:46:48.655759 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:46:48.656054 (+ 295us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:46:48.656245 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:46:48.656249 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:46:48.657025 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:46:48.657026 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:46:48.657858 (+ 832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:46:48.657860 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:46:48.658332 (+ 472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:46:48.658332 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:46:48.658470 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:46:48.659050 (+ 580us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:46:48.659059 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:46:48.660657 (+ 1598us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:46:48.662411 (+ 1754us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:46:48.662421 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:46:48.662423 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:46:48.662665 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:46:48.662906 (+ 241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:46:48.662907 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:46:48.662907 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:46:48.663025 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:46:48.663292 (+ 267us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:46:48.663299 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:46:48.663447 (+ 148us) client_negotiation.cc:769] Sending connection context
0806 03:46:48.663555 (+ 108us) client_negotiation.cc:240] Negotiation successful
0806 03:46:48.663721 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":82,"threads_started":1}
I20260806 03:46:48.664557 13309 heartbeater.cc:347] Connected to a master server at 127.11.253.254:45631
I20260806 03:46:48.664670 13309 heartbeater.cc:464] Registering TS with master...
I20260806 03:46:48.664871 13309 heartbeater.cc:511] Master 127.11.253.254:45631 requested a full tablet report, sending...
I20260806 03:46:48.665429 13037 ts_manager.cc:194] Registered new tserver with Master: 14542d79c36443f08164ae20dd0762fd (127.11.253.193:44463)
I20260806 03:46:48.666209 13037 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.253.193@KRBTEST.COM'} at 127.11.253.193:34747
WARNING: no policy specified for kudu/127.11.253.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.253.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.253.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.194.keytab.
Entry for principal kudu/127.11.253.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.194.keytab.
I20260806 03:46:48.689455 12279 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskToVWRe/build/release/bin/kudu
/tmp/dist-test-taskToVWRe/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785988004637995-12279-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785988004637995-12279-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785988004637995-12279-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785988004637995-12279-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.253.194:0
--local_ip_for_outbound_sockets=127.11.253.194
--webserver_interface=127.11.253.194
--webserver_port=0
--tserver_master_addrs=127.11.253.254:45631
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.194.keytab
--principal=kudu/127.11.253.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.253.212:44585
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785988004637995-12279-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785988004637995-12279-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785988004637995-12279-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:46:48.773145 13316 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:46:48.773397 13316 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:46:48.773437 13316 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:46:48.774626 13316 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:46:48.774718 13316 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:46:48.774766 13316 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.253.194
I20260806 03:46:48.776310 13316 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.253.212:44585
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785988004637995-12279-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785988004637995-12279-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.194.keytab
--principal=kudu/127.11.253.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.253.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785988004637995-12279-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.253.194
--webserver_port=0
--tserver_master_addrs=127.11.253.254:45631
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13316
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.253.194
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785988004637995-12279-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:46:48.776593 13316 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:46:48.776844 13316 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:46:48.781981 13323 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:46:48.782038 13321 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:46:48.782078 13316 server_base.cc:1034] running on GCE node
W20260806 03:46:48.781988 13326 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:46:48.782478 13316 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:46:48.782671 13316 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:46:48.783883 13316 hybrid_clock.cc:648] HybridClock initialized: now 1785988008783869 us; error 29 us; skew 500 ppm
Aug 06 03:46:48 dist-test-slave-6ntk krb5kdc[12972](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988008, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.253.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:46:48.786477 13316 init.cc:377] Logged in from keytab as kudu/127.11.253.194@KRBTEST.COM (short username kudu)
I20260806 03:46:48.786957 13316 webserver.cc:460] Webserver started at http://127.11.253.194:44027/ using document root <none> and password file <none>
I20260806 03:46:48.787221 13316 fs_manager.cc:359] Metadata directory not provided
I20260806 03:46:48.787310 13316 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:46:48.787470 13316 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:46:48.788517 13316 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785988004637995-12279-0/minicluster-data/ts-1/data/instance:
uuid: "b93c4d1b5c984170a89ec22de368c525"
format_stamp: "Formatted at 2026-08-06 03:46:48 on dist-test-slave-6ntk"
server_key: "a33b89a4a9d762c9bc2133200369c893"
server_key_iv: "05953ae68102d87ff53f40bca169a357"
server_key_version: "encryptionkey@0"
I20260806 03:46:48.788889 13316 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785988004637995-12279-0/minicluster-data/ts-1/wal/instance:
uuid: "b93c4d1b5c984170a89ec22de368c525"
format_stamp: "Formatted at 2026-08-06 03:46:48 on dist-test-slave-6ntk"
server_key: "a33b89a4a9d762c9bc2133200369c893"
server_key_iv: "05953ae68102d87ff53f40bca169a357"
server_key_version: "encryptionkey@0"
I20260806 03:46:48.790843 13316 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.004s
I20260806 03:46:48.791671 13337 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:46:48.791851 13316 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 03:46:48.791963 13316 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785988004637995-12279-0/minicluster-data/ts-1/data,/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785988004637995-12279-0/minicluster-data/ts-1/wal
uuid: "b93c4d1b5c984170a89ec22de368c525"
format_stamp: "Formatted at 2026-08-06 03:46:48 on dist-test-slave-6ntk"
server_key: "a33b89a4a9d762c9bc2133200369c893"
server_key_iv: "05953ae68102d87ff53f40bca169a357"
server_key_version: "encryptionkey@0"
I20260806 03:46:48.792052 13316 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785988004637995-12279-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785988004637995-12279-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785988004637995-12279-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:46:48.816039 13316 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:46:48.818578 13316 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:46:48.818756 13316 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:46:48.819031 13316 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 03:46:48.819365 13316 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 03:46:48.819423 13316 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 03:46:48.819458 13316 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 03:46:48.819479 13316 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 03:46:48.826342 13316 rpc_server.cc:307] RPC server started. Bound to: 127.11.253.194:33205
I20260806 03:46:48.826397 13489 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.253.194:33205 every 8 connection(s)
I20260806 03:46:48.826766 13316 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785988004637995-12279-0/minicluster-data/ts-1/data/info.pb
Aug 06 03:46:48 dist-test-slave-6ntk krb5kdc[12972](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988008, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.253.194@KRBTEST.COM for kudu/127.11.253.254@KRBTEST.COM
I20260806 03:46:48.835511 12279 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskToVWRe/build/release/bin/kudu as pid 13316
I20260806 03:46:48.835615 12279 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785988004637995-12279-0/minicluster-data/ts-1/wal/instance
I20260806 03:46:48.835820 12279 external_mini_cluster.cc:1428] Setting key 8911a38e83fd48e3960b190a2943e2b9
I20260806 03:46:48.837062 13124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:46:48.827608 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.253.194:53285 (local address 127.11.253.254:45631)
0806 03:46:48.827658 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0806 03:46:48.827660 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:46:48.827870 (+ 210us) server_negotiation.cc:407] Connection header received
0806 03:46:48.828834 (+ 964us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:46:48.828835 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:46:48.828846 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:46:48.828906 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:46:48.829534 (+ 628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:46:48.830133 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:46:48.831224 (+ 1091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:46:48.831451 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:46:48.831505 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:46:48.833590 (+ 2085us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:46:48.833591 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:46:48.833591 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:46:48.833599 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:46:48.835161 (+ 1562us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:46:48.835760 (+ 599us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:46:48.835762 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:46:48.835762 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:46:48.835889 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:46:48.836210 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:46:48.836212 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:46:48.836212 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:46:48.836369 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:46:48.836438 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0806 03:46:48.836823 (+ 385us) server_negotiation.cc:299] Negotiation successful
0806 03:46:48.836923 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260806 03:46:48.837137 13496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:46:48.827588 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:45631 (local address 127.11.253.194:53285)
0806 03:46:48.827743 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0806 03:46:48.827780 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0806 03:46:48.828701 (+ 921us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:46:48.828986 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:46:48.828995 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:46:48.829256 (+ 261us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:46:48.829457 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:46:48.829461 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:46:48.830211 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:46:48.830213 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:46:48.831154 (+ 941us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:46:48.831156 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:46:48.831525 (+ 369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:46:48.831525 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:46:48.831624 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:46:48.832124 (+ 500us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:46:48.832132 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:46:48.833507 (+ 1375us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:46:48.835292 (+ 1785us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:46:48.835301 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:46:48.835303 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:46:48.835651 (+ 348us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:46:48.835996 (+ 345us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:46:48.835997 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:46:48.835998 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:46:48.836115 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:46:48.836457 (+ 342us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:46:48.836463 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:46:48.836653 (+ 190us) client_negotiation.cc:769] Sending connection context
0806 03:46:48.836780 (+ 127us) client_negotiation.cc:240] Negotiation successful
0806 03:46:48.836927 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":81,"threads_started":1}
I20260806 03:46:48.837697 13490 heartbeater.cc:347] Connected to a master server at 127.11.253.254:45631
I20260806 03:46:48.837798 13490 heartbeater.cc:464] Registering TS with master...
I20260806 03:46:48.838038 13490 heartbeater.cc:511] Master 127.11.253.254:45631 requested a full tablet report, sending...
I20260806 03:46:48.838532 13037 ts_manager.cc:194] Registered new tserver with Master: b93c4d1b5c984170a89ec22de368c525 (127.11.253.194:33205)
I20260806 03:46:48.839114 13037 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.253.194@KRBTEST.COM'} at 127.11.253.194:53285
WARNING: no policy specified for kudu/127.11.253.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.253.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.253.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.195.keytab.
Entry for principal kudu/127.11.253.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.195.keytab.
I20260806 03:46:48.864584 12279 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskToVWRe/build/release/bin/kudu
/tmp/dist-test-taskToVWRe/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785988004637995-12279-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785988004637995-12279-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785988004637995-12279-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785988004637995-12279-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.253.195:0
--local_ip_for_outbound_sockets=127.11.253.195
--webserver_interface=127.11.253.195
--webserver_port=0
--tserver_master_addrs=127.11.253.254:45631
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.195.keytab
--principal=kudu/127.11.253.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.253.212:44585
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785988004637995-12279-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785988004637995-12279-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785988004637995-12279-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:46:48.951049 13500 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:46:48.951272 13500 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:46:48.951296 13500 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:46:48.952639 13500 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:46:48.952687 13500 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:46:48.952739 13500 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.253.195
I20260806 03:46:48.954336 13500 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.253.212:44585
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785988004637995-12279-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785988004637995-12279-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.195.keytab
--principal=kudu/127.11.253.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.253.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785988004637995-12279-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.253.195
--webserver_port=0
--tserver_master_addrs=127.11.253.254:45631
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13500
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.253.195
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785988004637995-12279-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:46:48.954568 13500 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:46:48.954807 13500 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:46:48.960152 13509 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:46:48.960201 13500 server_base.cc:1034] running on GCE node
W20260806 03:46:48.960177 13505 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:46:48.960137 13506 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:46:48.960623 13500 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:46:48.960878 13500 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:46:48.962141 13500 hybrid_clock.cc:648] HybridClock initialized: now 1785988008962141 us; error 47 us; skew 500 ppm
Aug 06 03:46:48 dist-test-slave-6ntk krb5kdc[12972](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988008, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.253.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:46:48.965173 13500 init.cc:377] Logged in from keytab as kudu/127.11.253.195@KRBTEST.COM (short username kudu)
I20260806 03:46:48.965818 13500 webserver.cc:460] Webserver started at http://127.11.253.195:33113/ using document root <none> and password file <none>
I20260806 03:46:48.966120 13500 fs_manager.cc:359] Metadata directory not provided
I20260806 03:46:48.966198 13500 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:46:48.966382 13500 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:46:48.967576 13500 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785988004637995-12279-0/minicluster-data/ts-2/data/instance:
uuid: "91a848d7e413450a8cba7d934fa725b9"
format_stamp: "Formatted at 2026-08-06 03:46:48 on dist-test-slave-6ntk"
server_key: "2728ca0790e3233306623d58b55c330d"
server_key_iv: "122863ead15f44a81f7307a530099708"
server_key_version: "encryptionkey@0"
I20260806 03:46:48.967970 13500 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785988004637995-12279-0/minicluster-data/ts-2/wal/instance:
uuid: "91a848d7e413450a8cba7d934fa725b9"
format_stamp: "Formatted at 2026-08-06 03:46:48 on dist-test-slave-6ntk"
server_key: "2728ca0790e3233306623d58b55c330d"
server_key_iv: "122863ead15f44a81f7307a530099708"
server_key_version: "encryptionkey@0"
I20260806 03:46:48.969414 13500 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 03:46:48.970288 13517 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:46:48.970489 13500 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260806 03:46:48.970609 13500 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785988004637995-12279-0/minicluster-data/ts-2/data,/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785988004637995-12279-0/minicluster-data/ts-2/wal
uuid: "91a848d7e413450a8cba7d934fa725b9"
format_stamp: "Formatted at 2026-08-06 03:46:48 on dist-test-slave-6ntk"
server_key: "2728ca0790e3233306623d58b55c330d"
server_key_iv: "122863ead15f44a81f7307a530099708"
server_key_version: "encryptionkey@0"
I20260806 03:46:48.970690 13500 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785988004637995-12279-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785988004637995-12279-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785988004637995-12279-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:46:48.984009 13500 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:46:48.986877 13500 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:46:48.987061 13500 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:46:48.987331 13500 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 03:46:48.987680 13500 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 03:46:48.987738 13500 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 03:46:48.987777 13500 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 03:46:48.987810 13500 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 03:46:48.994671 13500 rpc_server.cc:307] RPC server started. Bound to: 127.11.253.195:33545
I20260806 03:46:48.994728 13661 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.253.195:33545 every 8 connection(s)
I20260806 03:46:48.995102 13500 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785988004637995-12279-0/minicluster-data/ts-2/data/info.pb
I20260806 03:46:49.000190 12279 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskToVWRe/build/release/bin/kudu as pid 13500
I20260806 03:46:49.000300 12279 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785988004637995-12279-0/minicluster-data/ts-2/wal/instance
I20260806 03:46:49.000450 12279 external_mini_cluster.cc:1428] Setting key 0d02e02dbac909192c4817729f761927
Aug 06 03:46:48 dist-test-slave-6ntk krb5kdc[12972](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988008, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.253.195@KRBTEST.COM for kudu/127.11.253.254@KRBTEST.COM
I20260806 03:46:49.005095 13124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:46:48.995808 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.253.195:35383 (local address 127.11.253.254:45631)
0806 03:46:48.995886 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0806 03:46:48.995888 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:46:48.996134 (+ 246us) server_negotiation.cc:407] Connection header received
0806 03:46:48.997105 (+ 971us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:46:48.997106 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:46:48.997117 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:46:48.997159 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:46:48.997796 (+ 637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:46:48.998333 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:46:48.999407 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:46:48.999584 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:46:48.999633 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:46:49.002222 (+ 2589us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:46:49.002223 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:46:49.002224 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:46:49.002230 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:46:49.003524 (+ 1294us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:46:49.003920 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:46:49.003920 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:46:49.003921 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:46:49.003972 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:46:49.004374 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:46:49.004375 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:46:49.004376 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:46:49.004514 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:46:49.004601 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0806 03:46:49.004896 (+ 295us) server_negotiation.cc:299] Negotiation successful
0806 03:46:49.004989 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260806 03:46:49.005380 13665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:46:48.995806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:45631 (local address 127.11.253.195:35383)
0806 03:46:48.996001 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0806 03:46:48.996040 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0806 03:46:48.996999 (+ 959us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:46:48.997234 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:46:48.997238 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:46:48.997513 (+ 275us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:46:48.997717 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:46:48.997722 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:46:48.998423 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:46:48.998424 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:46:48.999330 (+ 906us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:46:48.999333 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:46:48.999660 (+ 327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:46:48.999661 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:46:48.999767 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:46:49.000398 (+ 631us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:46:49.000407 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:46:49.002138 (+ 1731us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:46:49.003605 (+ 1467us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:46:49.003611 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:46:49.003613 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:46:49.003849 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:46:49.004033 (+ 184us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:46:49.004034 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:46:49.004034 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:46:49.004199 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:46:49.004604 (+ 405us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:46:49.004612 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:46:49.004746 (+ 134us) client_negotiation.cc:769] Sending connection context
0806 03:46:49.004870 (+ 124us) client_negotiation.cc:240] Negotiation successful
0806 03:46:49.005037 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":106,"threads_started":1}
I20260806 03:46:49.006027 13663 heartbeater.cc:347] Connected to a master server at 127.11.253.254:45631
I20260806 03:46:49.006134 13663 heartbeater.cc:464] Registering TS with master...
I20260806 03:46:49.006323 13663 heartbeater.cc:511] Master 127.11.253.254:45631 requested a full tablet report, sending...
I20260806 03:46:49.006738 13037 ts_manager.cc:194] Registered new tserver with Master: 91a848d7e413450a8cba7d934fa725b9 (127.11.253.195:33545)
I20260806 03:46:49.007134 13037 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.253.195@KRBTEST.COM'} at 127.11.253.195:35383
I20260806 03:46:49.012367 12279 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 03:46:49.020062 13124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:46:49.014065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60026 (local address 127.11.253.254:45631)
0806 03:46:49.014125 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0806 03:46:49.014127 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:46:49.014212 (+ 85us) server_negotiation.cc:407] Connection header received
0806 03:46:49.014298 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:46:49.014299 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:46:49.014308 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:46:49.014368 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:46:49.015122 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:46:49.015605 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:46:49.016402 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:46:49.016584 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:46:49.016640 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:46:49.017823 (+ 1183us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:46:49.017824 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:46:49.017825 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:46:49.017831 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:46:49.019086 (+ 1255us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:46:49.019409 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:46:49.019409 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:46:49.019410 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:46:49.019460 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:46:49.019625 (+ 165us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:46:49.019626 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:46:49.019626 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:46:49.019760 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:46:49.019866 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0806 03:46:49.019937 (+ 71us) server_negotiation.cc:299] Negotiation successful
0806 03:46:49.019998 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260806 03:46:49.022279 13037 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60026:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 03:46:49.023885 13037 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 03:46:49.030962 13676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:46:49.026621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.195:33545 (local address 127.0.0.1:40704)
0806 03:46:49.026773 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0806 03:46:49.026818 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 03:46:49.026887 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:46:49.027336 (+ 449us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:46:49.027340 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:46:49.027343 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 03:46:49.027442 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:46:49.027445 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:46:49.028525 (+ 1080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:46:49.028526 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:46:49.029643 (+ 1117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:46:49.029645 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:46:49.030632 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:46:49.030633 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:46:49.030718 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:46:49.030734 (+ 16us) client_negotiation.cc:769] Sending connection context
0806 03:46:49.030817 (+ 83us) client_negotiation.cc:240] Negotiation successful
0806 03:46:49.030874 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":92,"thread_start_us":70,"threads_started":1}
I20260806 03:46:49.030975 13681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:46:49.026848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40704 (local address 127.11.253.195:33545)
0806 03:46:49.027092 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0806 03:46:49.027097 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 03:46:49.027108 (+ 11us) server_negotiation.cc:407] Connection header received
0806 03:46:49.027158 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:46:49.027161 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:46:49.027175 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:46:49.027325 (+ 150us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 03:46:49.027550 (+ 225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:46:49.028421 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:46:49.029740 (+ 1319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:46:49.030492 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:46:49.030603 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:46:49.030665 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0806 03:46:49.030840 (+ 175us) server_negotiation.cc:299] Negotiation successful
0806 03:46:49.030894 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":209,"thread_start_us":106,"threads_started":1}
I20260806 03:46:49.031638 13677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:46:49.026630 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.194:33205 (local address 127.0.0.1:50590)
0806 03:46:49.026790 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0806 03:46:49.026818 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0806 03:46:49.026887 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:46:49.027634 (+ 747us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:46:49.027635 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:46:49.027637 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 03:46:49.027677 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:46:49.027678 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:46:49.029360 (+ 1682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:46:49.029362 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:46:49.030426 (+ 1064us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:46:49.030429 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:46:49.031383 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:46:49.031384 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:46:49.031454 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:46:49.031458 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 03:46:49.031523 (+ 65us) client_negotiation.cc:240] Negotiation successful
0806 03:46:49.031564 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":107,"threads_started":1}
I20260806 03:46:49.031657 13683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:46:49.027086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50590 (local address 127.11.253.194:33205)
0806 03:46:49.027329 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0806 03:46:49.027334 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 03:46:49.027346 (+ 12us) server_negotiation.cc:407] Connection header received
0806 03:46:49.027392 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:46:49.027394 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:46:49.027412 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:46:49.027525 (+ 113us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 03:46:49.028008 (+ 483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:46:49.029242 (+ 1234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:46:49.030520 (+ 1278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:46:49.031283 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:46:49.031383 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:46:49.031442 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0806 03:46:49.031538 (+ 96us) server_negotiation.cc:299] Negotiation successful
0806 03:46:49.031579 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":198,"thread_start_us":99,"threads_started":1}
I20260806 03:46:49.031847 13678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:46:49.026879 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.193:44463 (local address 127.0.0.1:53316)
0806 03:46:49.027413 (+ 534us) negotiation.cc:107] Waiting for socket to connect
0806 03:46:49.027423 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 03:46:49.027522 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:46:49.027755 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:46:49.027755 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:46:49.027757 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 03:46:49.027791 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:46:49.027792 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:46:49.029242 (+ 1450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:46:49.029243 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:46:49.030491 (+ 1248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:46:49.030494 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:46:49.031628 (+ 1134us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:46:49.031629 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:46:49.031688 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:46:49.031692 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 03:46:49.031740 (+ 48us) client_negotiation.cc:240] Negotiation successful
0806 03:46:49.031780 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":515,"thread_start_us":75,"threads_started":1}
I20260806 03:46:49.032136 13684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:46:49.027136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53316 (local address 127.11.253.193:44463)
0806 03:46:49.027347 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0806 03:46:49.027353 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 03:46:49.027518 (+ 165us) server_negotiation.cc:407] Connection header received
0806 03:46:49.027634 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:46:49.027637 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:46:49.027655 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:46:49.027746 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 03:46:49.027904 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:46:49.029127 (+ 1223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:46:49.030590 (+ 1463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:46:49.031468 (+ 878us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:46:49.031538 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:46:49.031593 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0806 03:46:49.032009 (+ 416us) server_negotiation.cc:299] Negotiation successful
0806 03:46:49.032060 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":177,"thread_start_us":83,"threads_started":1}
I20260806 03:46:49.032051 13573 tablet_service.cc:1467] Processing CreateTablet for tablet 88506040f92747a0beb15ebe5135182a (DEFAULT_TABLE table=test-table [id=fdbbbdb89bf247b7a1bf0c8c3f4d93dd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 03:46:49.032457 13573 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 88506040f92747a0beb15ebe5135182a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:46:49.032486 13402 tablet_service.cc:1467] Processing CreateTablet for tablet 88506040f92747a0beb15ebe5135182a (DEFAULT_TABLE table=test-table [id=fdbbbdb89bf247b7a1bf0c8c3f4d93dd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 03:46:49.032752 13402 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 88506040f92747a0beb15ebe5135182a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:46:49.032728 13218 tablet_service.cc:1467] Processing CreateTablet for tablet 88506040f92747a0beb15ebe5135182a (DEFAULT_TABLE table=test-table [id=fdbbbdb89bf247b7a1bf0c8c3f4d93dd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 03:46:49.033037 13218 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 88506040f92747a0beb15ebe5135182a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:46:49.035686 13685 tablet_bootstrap.cc:492] T 88506040f92747a0beb15ebe5135182a P b93c4d1b5c984170a89ec22de368c525: Bootstrap starting.
I20260806 03:46:49.035900 13686 tablet_bootstrap.cc:492] T 88506040f92747a0beb15ebe5135182a P 91a848d7e413450a8cba7d934fa725b9: Bootstrap starting.
I20260806 03:46:49.036460 13685 tablet_bootstrap.cc:654] T 88506040f92747a0beb15ebe5135182a P b93c4d1b5c984170a89ec22de368c525: Neither blocks nor log segments found. Creating new log.
I20260806 03:46:49.036532 13688 tablet_bootstrap.cc:492] T 88506040f92747a0beb15ebe5135182a P 14542d79c36443f08164ae20dd0762fd: Bootstrap starting.
I20260806 03:46:49.036684 13686 tablet_bootstrap.cc:654] T 88506040f92747a0beb15ebe5135182a P 91a848d7e413450a8cba7d934fa725b9: Neither blocks nor log segments found. Creating new log.
I20260806 03:46:49.037109 13688 tablet_bootstrap.cc:654] T 88506040f92747a0beb15ebe5135182a P 14542d79c36443f08164ae20dd0762fd: Neither blocks nor log segments found. Creating new log.
I20260806 03:46:49.037261 13685 log.cc:824] T 88506040f92747a0beb15ebe5135182a P b93c4d1b5c984170a89ec22de368c525: Log is configured to *not* fsync() on all Append() calls
I20260806 03:46:49.037413 13686 log.cc:824] T 88506040f92747a0beb15ebe5135182a P 91a848d7e413450a8cba7d934fa725b9: Log is configured to *not* fsync() on all Append() calls
I20260806 03:46:49.037786 13688 log.cc:824] T 88506040f92747a0beb15ebe5135182a P 14542d79c36443f08164ae20dd0762fd: Log is configured to *not* fsync() on all Append() calls
I20260806 03:46:49.038241 13686 tablet_bootstrap.cc:492] T 88506040f92747a0beb15ebe5135182a P 91a848d7e413450a8cba7d934fa725b9: No bootstrap required, opened a new log
I20260806 03:46:49.038256 13685 tablet_bootstrap.cc:492] T 88506040f92747a0beb15ebe5135182a P b93c4d1b5c984170a89ec22de368c525: No bootstrap required, opened a new log
I20260806 03:46:49.038345 13686 ts_tablet_manager.cc:1397] T 88506040f92747a0beb15ebe5135182a P 91a848d7e413450a8cba7d934fa725b9: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 03:46:49.038415 13685 ts_tablet_manager.cc:1397] T 88506040f92747a0beb15ebe5135182a P b93c4d1b5c984170a89ec22de368c525: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 03:46:49.038939 13688 tablet_bootstrap.cc:492] T 88506040f92747a0beb15ebe5135182a P 14542d79c36443f08164ae20dd0762fd: No bootstrap required, opened a new log
I20260806 03:46:49.039019 13688 ts_tablet_manager.cc:1397] T 88506040f92747a0beb15ebe5135182a P 14542d79c36443f08164ae20dd0762fd: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 03:46:49.039901 13686 raft_consensus.cc:348] T 88506040f92747a0beb15ebe5135182a P 91a848d7e413450a8cba7d934fa725b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91a848d7e413450a8cba7d934fa725b9" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 33545 } } peers { permanent_uuid: "b93c4d1b5c984170a89ec22de368c525" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 33205 } } peers { permanent_uuid: "14542d79c36443f08164ae20dd0762fd" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 44463 } }
I20260806 03:46:49.040066 13686 raft_consensus.cc:374] T 88506040f92747a0beb15ebe5135182a P 91a848d7e413450a8cba7d934fa725b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:46:49.040112 13686 raft_consensus.cc:729] T 88506040f92747a0beb15ebe5135182a P 91a848d7e413450a8cba7d934fa725b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91a848d7e413450a8cba7d934fa725b9, State: Initialized, Role: FOLLOWER
I20260806 03:46:49.040246 13686 consensus_queue.cc:260] T 88506040f92747a0beb15ebe5135182a P 91a848d7e413450a8cba7d934fa725b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91a848d7e413450a8cba7d934fa725b9" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 33545 } } peers { permanent_uuid: "b93c4d1b5c984170a89ec22de368c525" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 33205 } } peers { permanent_uuid: "14542d79c36443f08164ae20dd0762fd" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 44463 } }
I20260806 03:46:49.040529 13686 ts_tablet_manager.cc:1428] T 88506040f92747a0beb15ebe5135182a P 91a848d7e413450a8cba7d934fa725b9: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 03:46:49.040676 13663 heartbeater.cc:503] Master 127.11.253.254:45631 was elected leader, sending a full tablet report...
I20260806 03:46:49.040697 13685 raft_consensus.cc:348] T 88506040f92747a0beb15ebe5135182a P b93c4d1b5c984170a89ec22de368c525 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91a848d7e413450a8cba7d934fa725b9" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 33545 } } peers { permanent_uuid: "b93c4d1b5c984170a89ec22de368c525" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 33205 } } peers { permanent_uuid: "14542d79c36443f08164ae20dd0762fd" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 44463 } }
I20260806 03:46:49.040887 13685 raft_consensus.cc:374] T 88506040f92747a0beb15ebe5135182a P b93c4d1b5c984170a89ec22de368c525 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:46:49.040940 13685 raft_consensus.cc:729] T 88506040f92747a0beb15ebe5135182a P b93c4d1b5c984170a89ec22de368c525 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b93c4d1b5c984170a89ec22de368c525, State: Initialized, Role: FOLLOWER
I20260806 03:46:49.041069 13685 consensus_queue.cc:260] T 88506040f92747a0beb15ebe5135182a P b93c4d1b5c984170a89ec22de368c525 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91a848d7e413450a8cba7d934fa725b9" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 33545 } } peers { permanent_uuid: "b93c4d1b5c984170a89ec22de368c525" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 33205 } } peers { permanent_uuid: "14542d79c36443f08164ae20dd0762fd" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 44463 } }
I20260806 03:46:49.041252 13688 raft_consensus.cc:348] T 88506040f92747a0beb15ebe5135182a P 14542d79c36443f08164ae20dd0762fd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91a848d7e413450a8cba7d934fa725b9" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 33545 } } peers { permanent_uuid: "b93c4d1b5c984170a89ec22de368c525" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 33205 } } peers { permanent_uuid: "14542d79c36443f08164ae20dd0762fd" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 44463 } }
I20260806 03:46:49.041388 13685 ts_tablet_manager.cc:1428] T 88506040f92747a0beb15ebe5135182a P b93c4d1b5c984170a89ec22de368c525: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260806 03:46:49.041393 13490 heartbeater.cc:503] Master 127.11.253.254:45631 was elected leader, sending a full tablet report...
I20260806 03:46:49.041451 13688 raft_consensus.cc:374] T 88506040f92747a0beb15ebe5135182a P 14542d79c36443f08164ae20dd0762fd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:46:49.041494 13688 raft_consensus.cc:729] T 88506040f92747a0beb15ebe5135182a P 14542d79c36443f08164ae20dd0762fd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 14542d79c36443f08164ae20dd0762fd, State: Initialized, Role: FOLLOWER
I20260806 03:46:49.041613 13688 consensus_queue.cc:260] T 88506040f92747a0beb15ebe5135182a P 14542d79c36443f08164ae20dd0762fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91a848d7e413450a8cba7d934fa725b9" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 33545 } } peers { permanent_uuid: "b93c4d1b5c984170a89ec22de368c525" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 33205 } } peers { permanent_uuid: "14542d79c36443f08164ae20dd0762fd" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 44463 } }
I20260806 03:46:49.041937 13688 ts_tablet_manager.cc:1428] T 88506040f92747a0beb15ebe5135182a P 14542d79c36443f08164ae20dd0762fd: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 03:46:49.042078 13309 heartbeater.cc:503] Master 127.11.253.254:45631 was elected leader, sending a full tablet report...
W20260806 03:46:49.077445 13494 tablet.cc:2369] T 88506040f92747a0beb15ebe5135182a P b93c4d1b5c984170a89ec22de368c525: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 03:46:49.149297 13694 raft_consensus.cc:482] T 88506040f92747a0beb15ebe5135182a P 14542d79c36443f08164ae20dd0762fd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 03:46:49.149452 13694 raft_consensus.cc:504] T 88506040f92747a0beb15ebe5135182a P 14542d79c36443f08164ae20dd0762fd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91a848d7e413450a8cba7d934fa725b9" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 33545 } } peers { permanent_uuid: "b93c4d1b5c984170a89ec22de368c525" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 33205 } } peers { permanent_uuid: "14542d79c36443f08164ae20dd0762fd" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 44463 } }
I20260806 03:46:49.150143 13694 leader_election.cc:288] T 88506040f92747a0beb15ebe5135182a P 14542d79c36443f08164ae20dd0762fd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 91a848d7e413450a8cba7d934fa725b9 (127.11.253.195:33545), b93c4d1b5c984170a89ec22de368c525 (127.11.253.194:33205)
I20260806 03:46:49.153214 13683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:46:49.150534 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.253.193:52821 (local address 127.11.253.194:33205)
0806 03:46:49.150586 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0806 03:46:49.150588 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:46:49.150603 (+ 15us) server_negotiation.cc:407] Connection header received
0806 03:46:49.150640 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:46:49.150642 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:46:49.150652 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:46:49.150758 (+ 106us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 03:46:49.150947 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:46:49.151434 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:46:49.152425 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:46:49.152837 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:46:49.152928 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:46:49.152939 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0806 03:46:49.153106 (+ 167us) server_negotiation.cc:299] Negotiation successful
0806 03:46:49.153145 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260806 03:46:49.153223 13311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:46:49.150458 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.194:33205 (local address 127.11.253.193:52821)
0806 03:46:49.150506 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0806 03:46:49.150520 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 03:46:49.150575 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:46:49.150781 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:46:49.150783 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:46:49.150785 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 03:46:49.150866 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:46:49.150868 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:46:49.151525 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:46:49.151526 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:46:49.152298 (+ 772us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:46:49.152300 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:46:49.152927 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:46:49.152929 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:46:49.153019 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:46:49.153022 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 03:46:49.153087 (+ 65us) client_negotiation.cc:240] Negotiation successful
0806 03:46:49.153128 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25}
I20260806 03:46:49.153426 13695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:46:49.150469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.195:33545 (local address 127.11.253.193:60825)
0806 03:46:49.150665 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0806 03:46:49.150690 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0806 03:46:49.150764 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:46:49.150955 (+ 191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:46:49.150955 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:46:49.150957 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 03:46:49.151053 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:46:49.151055 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:46:49.151734 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:46:49.151735 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:46:49.152505 (+ 770us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:46:49.152508 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:46:49.153123 (+ 615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:46:49.153124 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:46:49.153212 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:46:49.153216 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 03:46:49.153288 (+ 72us) client_negotiation.cc:240] Negotiation successful
0806 03:46:49.153331 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":186,"mutex_wait_us":47,"thread_start_us":94,"threads_started":1}
I20260806 03:46:49.153417 13681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:46:49.150561 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.253.193:60825 (local address 127.11.253.195:33545)
0806 03:46:49.150619 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0806 03:46:49.150621 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:46:49.150766 (+ 145us) server_negotiation.cc:407] Connection header received
0806 03:46:49.150827 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:46:49.150828 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:46:49.150840 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:46:49.150944 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 03:46:49.151137 (+ 193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:46:49.151610 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:46:49.152617 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:46:49.153025 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:46:49.153122 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:46:49.153131 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0806 03:46:49.153309 (+ 178us) server_negotiation.cc:299] Negotiation successful
0806 03:46:49.153352 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260806 03:46:49.153693 13429 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "88506040f92747a0beb15ebe5135182a" candidate_uuid: "14542d79c36443f08164ae20dd0762fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b93c4d1b5c984170a89ec22de368c525" is_pre_election: true
I20260806 03:46:49.153975 13429 raft_consensus.cc:2454] T 88506040f92747a0beb15ebe5135182a P b93c4d1b5c984170a89ec22de368c525 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 14542d79c36443f08164ae20dd0762fd in term 0.
W20260806 03:46:49.154069 13310 tablet.cc:2369] T 88506040f92747a0beb15ebe5135182a P 14542d79c36443f08164ae20dd0762fd: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 03:46:49.154140 13599 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "88506040f92747a0beb15ebe5135182a" candidate_uuid: "14542d79c36443f08164ae20dd0762fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91a848d7e413450a8cba7d934fa725b9" is_pre_election: true
I20260806 03:46:49.154280 13158 leader_election.cc:302] T 88506040f92747a0beb15ebe5135182a P 14542d79c36443f08164ae20dd0762fd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 14542d79c36443f08164ae20dd0762fd, b93c4d1b5c984170a89ec22de368c525; no voters:
I20260806 03:46:49.154311 13599 raft_consensus.cc:2454] T 88506040f92747a0beb15ebe5135182a P 91a848d7e413450a8cba7d934fa725b9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 14542d79c36443f08164ae20dd0762fd in term 0.
I20260806 03:46:49.154425 13694 raft_consensus.cc:2781] T 88506040f92747a0beb15ebe5135182a P 14542d79c36443f08164ae20dd0762fd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 03:46:49.154480 13694 raft_consensus.cc:482] T 88506040f92747a0beb15ebe5135182a P 14542d79c36443f08164ae20dd0762fd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 03:46:49.154507 13694 raft_consensus.cc:3037] T 88506040f92747a0beb15ebe5135182a P 14542d79c36443f08164ae20dd0762fd [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:46:49.155366 13694 raft_consensus.cc:504] T 88506040f92747a0beb15ebe5135182a P 14542d79c36443f08164ae20dd0762fd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91a848d7e413450a8cba7d934fa725b9" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 33545 } } peers { permanent_uuid: "b93c4d1b5c984170a89ec22de368c525" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 33205 } } peers { permanent_uuid: "14542d79c36443f08164ae20dd0762fd" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 44463 } }
I20260806 03:46:49.155529 13694 leader_election.cc:288] T 88506040f92747a0beb15ebe5135182a P 14542d79c36443f08164ae20dd0762fd [CANDIDATE]: Term 1 election: Requested vote from peers 91a848d7e413450a8cba7d934fa725b9 (127.11.253.195:33545), b93c4d1b5c984170a89ec22de368c525 (127.11.253.194:33205)
I20260806 03:46:49.155725 13429 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "88506040f92747a0beb15ebe5135182a" candidate_uuid: "14542d79c36443f08164ae20dd0762fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b93c4d1b5c984170a89ec22de368c525"
I20260806 03:46:49.155819 13429 raft_consensus.cc:3037] T 88506040f92747a0beb15ebe5135182a P b93c4d1b5c984170a89ec22de368c525 [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:46:49.155834 13599 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "88506040f92747a0beb15ebe5135182a" candidate_uuid: "14542d79c36443f08164ae20dd0762fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91a848d7e413450a8cba7d934fa725b9"
I20260806 03:46:49.155951 13599 raft_consensus.cc:3037] T 88506040f92747a0beb15ebe5135182a P 91a848d7e413450a8cba7d934fa725b9 [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:46:49.156601 13429 raft_consensus.cc:2454] T 88506040f92747a0beb15ebe5135182a P b93c4d1b5c984170a89ec22de368c525 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 14542d79c36443f08164ae20dd0762fd in term 1.
I20260806 03:46:49.156795 13158 leader_election.cc:302] T 88506040f92747a0beb15ebe5135182a P 14542d79c36443f08164ae20dd0762fd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 14542d79c36443f08164ae20dd0762fd, b93c4d1b5c984170a89ec22de368c525; no voters:
I20260806 03:46:49.156916 13694 raft_consensus.cc:2781] T 88506040f92747a0beb15ebe5135182a P 14542d79c36443f08164ae20dd0762fd [term 1 FOLLOWER]: Leader election won for term 1
I20260806 03:46:49.157030 13599 raft_consensus.cc:2454] T 88506040f92747a0beb15ebe5135182a P 91a848d7e413450a8cba7d934fa725b9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 14542d79c36443f08164ae20dd0762fd in term 1.
I20260806 03:46:49.157138 13694 raft_consensus.cc:686] T 88506040f92747a0beb15ebe5135182a P 14542d79c36443f08164ae20dd0762fd [term 1 LEADER]: Becoming Leader. State: Replica: 14542d79c36443f08164ae20dd0762fd, State: Running, Role: LEADER
I20260806 03:46:49.157375 13694 consensus_queue.cc:237] T 88506040f92747a0beb15ebe5135182a P 14542d79c36443f08164ae20dd0762fd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91a848d7e413450a8cba7d934fa725b9" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 33545 } } peers { permanent_uuid: "b93c4d1b5c984170a89ec22de368c525" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 33205 } } peers { permanent_uuid: "14542d79c36443f08164ae20dd0762fd" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 44463 } }
I20260806 03:46:49.158381 13037 catalog_manager.cc:5495] T 88506040f92747a0beb15ebe5135182a P 14542d79c36443f08164ae20dd0762fd reported cstate change: term changed from 0 to 1, leader changed from <none> to 14542d79c36443f08164ae20dd0762fd (127.11.253.193). New cstate: current_term: 1 leader_uuid: "14542d79c36443f08164ae20dd0762fd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91a848d7e413450a8cba7d934fa725b9" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 33545 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b93c4d1b5c984170a89ec22de368c525" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 33205 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "14542d79c36443f08164ae20dd0762fd" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 44463 } health_report { overall_health: HEALTHY } } }
Aug 06 03:46:49 dist-test-slave-6ntk krb5kdc[12972](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988008, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.253.193@KRBTEST.COM
I20260806 03:46:49.180251 13684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:46:49.172662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53328 (local address 127.11.253.193:44463)
0806 03:46:49.172717 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0806 03:46:49.172719 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:46:49.172795 (+ 76us) server_negotiation.cc:407] Connection header received
0806 03:46:49.172853 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:46:49.172854 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:46:49.172866 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:46:49.172908 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:46:49.173521 (+ 613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:46:49.174121 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:46:49.174930 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:46:49.175118 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:46:49.175173 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:46:49.177604 (+ 2431us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:46:49.177618 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:46:49.177618 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:46:49.177625 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:46:49.179241 (+ 1616us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:46:49.179565 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:46:49.179568 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:46:49.179569 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:46:49.179640 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:46:49.179810 (+ 170us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:46:49.179810 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:46:49.179811 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:46:49.179952 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:46:49.180065 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0806 03:46:49.180117 (+ 52us) server_negotiation.cc:299] Negotiation successful
0806 03:46:49.180181 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
Aug 06 03:46:49 dist-test-slave-6ntk krb5kdc[12972](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988009, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 06 03:46:49 dist-test-slave-6ntk krb5kdc[12972](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988009, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.11.253.193@KRBTEST.COM
I20260806 03:46:49.208287 13684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:46:49.200758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53332 (local address 127.11.253.193:44463)
0806 03:46:49.200852 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0806 03:46:49.200855 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 03:46:49.200946 (+ 91us) server_negotiation.cc:407] Connection header received
0806 03:46:49.201049 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:46:49.201050 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:46:49.201060 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:46:49.201136 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:46:49.201732 (+ 596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:46:49.202313 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:46:49.203060 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:46:49.203279 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:46:49.203366 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:46:49.205681 (+ 2315us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:46:49.205682 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:46:49.205682 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:46:49.205691 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:46:49.207027 (+ 1336us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:46:49.207392 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:46:49.207393 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:46:49.207393 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:46:49.207456 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:46:49.207661 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:46:49.207662 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:46:49.207663 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:46:49.207771 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:46:49.207887 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0806 03:46:49.208087 (+ 200us) server_negotiation.cc:299] Negotiation successful
0806 03:46:49.208175 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
Aug 06 03:46:49 dist-test-slave-6ntk krb5kdc[12972](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988009, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 03:46:49 dist-test-slave-6ntk krb5kdc[12972](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988009, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.253.193@KRBTEST.COM
I20260806 03:46:49.234797 13684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:46:49.227143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53334 (local address 127.11.253.193:44463)
0806 03:46:49.227256 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0806 03:46:49.227258 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:46:49.227277 (+ 19us) server_negotiation.cc:407] Connection header received
0806 03:46:49.227327 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:46:49.227328 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:46:49.227337 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:46:49.227396 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:46:49.227951 (+ 555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:46:49.228425 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:46:49.229307 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:46:49.229560 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:46:49.229631 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:46:49.232129 (+ 2498us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:46:49.232130 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:46:49.232131 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:46:49.232138 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:46:49.233589 (+ 1451us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:46:49.234005 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:46:49.234006 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:46:49.234007 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:46:49.234062 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:46:49.234305 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:46:49.234306 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:46:49.234306 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:46:49.234404 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:46:49.234487 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0806 03:46:49.234668 (+ 181us) server_negotiation.cc:299] Negotiation successful
0806 03:46:49.234728 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
W20260806 03:46:49.235388 13190 tablet_service.cc:2908] Rejecting scan request for tablet 88506040f92747a0beb15ebe5135182a: Uninitialized: clean time has not yet been initialized
I20260806 03:46:49.236635 12279 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskToVWRe/build/release/bin/kudu with pid 13131
W20260806 03:46:49.245725 13664 tablet.cc:2369] T 88506040f92747a0beb15ebe5135182a P 91a848d7e413450a8cba7d934fa725b9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 03:46:49.246762 12279 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskToVWRe/build/release/bin/kudu with pid 13316
I20260806 03:46:49.256628 12279 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskToVWRe/build/release/bin/kudu with pid 13500
I20260806 03:46:49.267484 12279 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskToVWRe/build/release/bin/kudu with pid 12989
2026-08-06T03:46:49Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2296 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785988004637995-12279-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 03:46:49 dist-test-slave-6ntk krb5kdc[13717](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 03:46:49 dist-test-slave-6ntk krb5kdc[13717](info): set up 2 sockets
Aug 06 03:46:49 dist-test-slave-6ntk krb5kdc[13717](info): commencing operation
krb5kdc: starting...
W20260806 03:46:50.594956 12279 mini_kdc.cc:121] Time spent starting KDC: real 1.301s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785988004637995-12279-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785988004637995-12279-0/krb5kdc/test-user.keytab.
Aug 06 03:46:50 dist-test-slave-6ntk krb5kdc[13717](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988010, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T03:46:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T03:46:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.253.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.253.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.253.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.254.keytab.
Entry for principal kudu/127.11.253.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.254.keytab.
I20260806 03:46:50.724486 12279 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskToVWRe/build/release/bin/kudu
/tmp/dist-test-taskToVWRe/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785988004637995-12279-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785988004637995-12279-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785988004637995-12279-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785988004637995-12279-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.253.254:37929
--webserver_interface=127.11.253.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.254.keytab
--principal=kudu/127.11.253.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.253.212:39969
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.253.254:37929
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785988004637995-12279-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785988004637995-12279-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785988004637995-12279-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:46:50.808674 13739 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:46:50.808880 13739 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:46:50.808903 13739 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:46:50.810133 13739 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 03:46:50.810173 13739 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260806 03:46:50.810186 13739 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:46:50.810199 13739 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:46:50.810235 13739 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 03:46:50.810272 13739 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 03:46:50.811702 13739 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.253.212:39969
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785988004637995-12279-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785988004637995-12279-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.253.254:37929
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.254.keytab
--principal=kudu/127.11.253.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.253.254:37929
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785988004637995-12279-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.253.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13739
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785988004637995-12279-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:46:50.811940 13739 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:46:50.812187 13739 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:46:50.817220 13744 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:46:50.817178 13746 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:46:50.817396 13739 server_base.cc:1034] running on GCE node
W20260806 03:46:50.817188 13749 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:46:50.817974 13739 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:46:50.818279 13739 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:46:50.819463 13739 hybrid_clock.cc:648] HybridClock initialized: now 1785988010819441 us; error 37 us; skew 500 ppm
Aug 06 03:46:50 dist-test-slave-6ntk krb5kdc[13717](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988010, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.253.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:46:50.822178 13739 init.cc:377] Logged in from keytab as kudu/127.11.253.254@KRBTEST.COM (short username kudu)
I20260806 03:46:50.822710 13739 webserver.cc:460] Webserver started at http://127.11.253.254:45559/ using document root <none> and password file <none>
I20260806 03:46:50.822947 13739 fs_manager.cc:359] Metadata directory not provided
I20260806 03:46:50.823016 13739 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:46:50.823164 13739 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:46:50.824270 13739 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785988004637995-12279-0/minicluster-data/master-0/data/instance:
uuid: "2827edda0baf420a983b2c4a3c88ddaf"
format_stamp: "Formatted at 2026-08-06 03:46:50 on dist-test-slave-6ntk"
server_key: "f47cf4926973fc907b48362cc6c0a086"
server_key_iv: "18d67897a9d276d4dd56ce0334b514c0"
server_key_version: "encryptionkey@0"
I20260806 03:46:50.824666 13739 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785988004637995-12279-0/minicluster-data/master-0/wal/instance:
uuid: "2827edda0baf420a983b2c4a3c88ddaf"
format_stamp: "Formatted at 2026-08-06 03:46:50 on dist-test-slave-6ntk"
server_key: "f47cf4926973fc907b48362cc6c0a086"
server_key_iv: "18d67897a9d276d4dd56ce0334b514c0"
server_key_version: "encryptionkey@0"
I20260806 03:46:50.826148 13739 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 03:46:50.826972 13759 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:46:50.827143 13739 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 03:46:50.827255 13739 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785988004637995-12279-0/minicluster-data/master-0/data,/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785988004637995-12279-0/minicluster-data/master-0/wal
uuid: "2827edda0baf420a983b2c4a3c88ddaf"
format_stamp: "Formatted at 2026-08-06 03:46:50 on dist-test-slave-6ntk"
server_key: "f47cf4926973fc907b48362cc6c0a086"
server_key_iv: "18d67897a9d276d4dd56ce0334b514c0"
server_key_version: "encryptionkey@0"
I20260806 03:46:50.827330 13739 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785988004637995-12279-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785988004637995-12279-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785988004637995-12279-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:46:50.851043 13739 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:46:50.853472 13739 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:46:50.853646 13739 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:46:50.858757 13739 rpc_server.cc:307] RPC server started. Bound to: 127.11.253.254:37929
I20260806 03:46:50.858816 13846 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.253.254:37929 every 8 connection(s)
I20260806 03:46:50.859146 13739 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785988004637995-12279-0/minicluster-data/master-0/data/info.pb
I20260806 03:46:50.859769 13847 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:46:50.860018 12279 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskToVWRe/build/release/bin/kudu as pid 13739
I20260806 03:46:50.860122 12279 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785988004637995-12279-0/minicluster-data/master-0/wal/instance
I20260806 03:46:50.860306 12279 external_mini_cluster.cc:1428] Setting key de56deb84359d6ba51621c06ecea8aac
I20260806 03:46:50.862871 13847 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2827edda0baf420a983b2c4a3c88ddaf: Bootstrap starting.
I20260806 03:46:50.863534 13847 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2827edda0baf420a983b2c4a3c88ddaf: Neither blocks nor log segments found. Creating new log.
I20260806 03:46:50.863826 13847 log.cc:824] T 00000000000000000000000000000000 P 2827edda0baf420a983b2c4a3c88ddaf: Log is configured to *not* fsync() on all Append() calls
I20260806 03:46:50.864516 13847 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2827edda0baf420a983b2c4a3c88ddaf: No bootstrap required, opened a new log
Aug 06 03:46:50 dist-test-slave-6ntk krb5kdc[13717](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988010, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.253.254@KRBTEST.COM
I20260806 03:46:50.866901 13847 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2827edda0baf420a983b2c4a3c88ddaf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2827edda0baf420a983b2c4a3c88ddaf" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 37929 } }
I20260806 03:46:50.867087 13847 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2827edda0baf420a983b2c4a3c88ddaf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:46:50.867141 13847 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2827edda0baf420a983b2c4a3c88ddaf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2827edda0baf420a983b2c4a3c88ddaf, State: Initialized, Role: FOLLOWER
I20260806 03:46:50.867300 13847 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2827edda0baf420a983b2c4a3c88ddaf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2827edda0baf420a983b2c4a3c88ddaf" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 37929 } }
I20260806 03:46:50.867396 13847 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2827edda0baf420a983b2c4a3c88ddaf [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 03:46:50.867484 13847 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2827edda0baf420a983b2c4a3c88ddaf [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 03:46:50.867559 13847 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2827edda0baf420a983b2c4a3c88ddaf [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:46:50.868253 13847 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2827edda0baf420a983b2c4a3c88ddaf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2827edda0baf420a983b2c4a3c88ddaf" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 37929 } }
I20260806 03:46:50.868396 13847 leader_election.cc:302] T 00000000000000000000000000000000 P 2827edda0baf420a983b2c4a3c88ddaf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2827edda0baf420a983b2c4a3c88ddaf; no voters:
I20260806 03:46:50.868573 13847 leader_election.cc:288] T 00000000000000000000000000000000 P 2827edda0baf420a983b2c4a3c88ddaf [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 03:46:50.868640 13854 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2827edda0baf420a983b2c4a3c88ddaf [term 1 FOLLOWER]: Leader election won for term 1
I20260806 03:46:50.868804 13854 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2827edda0baf420a983b2c4a3c88ddaf [term 1 LEADER]: Becoming Leader. State: Replica: 2827edda0baf420a983b2c4a3c88ddaf, State: Running, Role: LEADER
I20260806 03:46:50.868904 13847 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2827edda0baf420a983b2c4a3c88ddaf [sys.catalog]: configured and running, proceeding with master startup.
I20260806 03:46:50.868997 13854 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2827edda0baf420a983b2c4a3c88ddaf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2827edda0baf420a983b2c4a3c88ddaf" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 37929 } }
I20260806 03:46:50.869444 13854 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2827edda0baf420a983b2c4a3c88ddaf [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2827edda0baf420a983b2c4a3c88ddaf. Latest consensus state: current_term: 1 leader_uuid: "2827edda0baf420a983b2c4a3c88ddaf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2827edda0baf420a983b2c4a3c88ddaf" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 37929 } } }
I20260806 03:46:50.869457 13856 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2827edda0baf420a983b2c4a3c88ddaf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2827edda0baf420a983b2c4a3c88ddaf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2827edda0baf420a983b2c4a3c88ddaf" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 37929 } } }
I20260806 03:46:50.869622 13856 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2827edda0baf420a983b2c4a3c88ddaf [sys.catalog]: This master's current role is: LEADER
I20260806 03:46:50.869887 13854 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2827edda0baf420a983b2c4a3c88ddaf [sys.catalog]: This master's current role is: LEADER
I20260806 03:46:50.869892 13865 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 03:46:50.870472 13865 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 03:46:50.871276 13852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:46:50.860846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58296 (local address 127.11.253.254:37929)
0806 03:46:50.861043 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0806 03:46:50.861051 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 03:46:50.861077 (+ 26us) server_negotiation.cc:407] Connection header received
0806 03:46:50.861510 (+ 433us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:46:50.861517 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:46:50.861561 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:46:50.861709 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:46:50.862413 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:46:50.863352 (+ 939us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:46:50.864226 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:46:50.864676 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:46:50.864801 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:46:50.867486 (+ 2685us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:46:50.867490 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:46:50.867490 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:46:50.867503 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:46:50.869566 (+ 2063us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:46:50.870033 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:46:50.870036 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:46:50.870036 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:46:50.870148 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:46:50.870358 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:46:50.870359 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:46:50.870359 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:46:50.870649 (+ 290us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:46:50.870804 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0806 03:46:50.870898 (+ 94us) server_negotiation.cc:299] Negotiation successful
0806 03:46:50.871035 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":86,"threads_started":1}
I20260806 03:46:50.871917 13876 mvcc.cc:205] Tried to move back new op lower bound from 7315406892529278976 to 7315406892520067072. Current Snapshot: MvccSnapshot[applied={T|T < 7315406892529278976}]
I20260806 03:46:50.872320 13865 catalog_manager.cc:1347] Generated new cluster ID: 6628a1d8de944ff3812c52d28635a4f9
I20260806 03:46:50.872373 13865 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 03:46:50.886564 13865 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 03:46:50.887141 13865 catalog_manager.cc:1504] Loading token signing keys...
I20260806 03:46:50.892800 13865 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2827edda0baf420a983b2c4a3c88ddaf: Generated new TSK 0
I20260806 03:46:50.892995 13865 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.253.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.253.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.253.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.193.keytab.
Entry for principal kudu/127.11.253.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.193.keytab.
I20260806 03:46:50.929229 12279 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskToVWRe/build/release/bin/kudu
/tmp/dist-test-taskToVWRe/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785988004637995-12279-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785988004637995-12279-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785988004637995-12279-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785988004637995-12279-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.253.193:0
--local_ip_for_outbound_sockets=127.11.253.193
--webserver_interface=127.11.253.193
--webserver_port=0
--tserver_master_addrs=127.11.253.254:37929
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.193.keytab
--principal=kudu/127.11.253.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.253.212:39969
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785988004637995-12279-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785988004637995-12279-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785988004637995-12279-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:46:51.014074 13881 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:46:51.014279 13881 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:46:51.014334 13881 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:46:51.015502 13881 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:46:51.015542 13881 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:46:51.015563 13881 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 03:46:51.015580 13881 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.253.193
I20260806 03:46:51.017709 13881 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.253.212:39969
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785988004637995-12279-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785988004637995-12279-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.193.keytab
--principal=kudu/127.11.253.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.253.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785988004637995-12279-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.253.193
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.11.253.254:37929
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13881
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.253.193
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785988004637995-12279-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:46:51.018016 13881 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:46:51.018280 13881 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:46:51.025031 13890 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:46:51.026814 13881 server_base.cc:1034] running on GCE node
W20260806 03:46:51.026889 13894 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:46:51.025069 13887 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:46:51.027215 13881 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:46:51.027484 13881 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:46:51.028656 13881 hybrid_clock.cc:648] HybridClock initialized: now 1785988011028639 us; error 36 us; skew 500 ppm
Aug 06 03:46:51 dist-test-slave-6ntk krb5kdc[13717](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988011, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.253.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:46:51.031921 13881 init.cc:377] Logged in from keytab as kudu/127.11.253.193@KRBTEST.COM (short username kudu)
I20260806 03:46:51.032439 13881 webserver.cc:460] Webserver started at http://127.11.253.193:35833/ using document root <none> and password file <none>
I20260806 03:46:51.032702 13881 fs_manager.cc:359] Metadata directory not provided
I20260806 03:46:51.032774 13881 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:46:51.032920 13881 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:46:51.034122 13881 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785988004637995-12279-0/minicluster-data/ts-0/data/instance:
uuid: "f499b5e4550248b6b408d516e5cea4c5"
format_stamp: "Formatted at 2026-08-06 03:46:51 on dist-test-slave-6ntk"
server_key: "ded0e72a62ad734bc28ca6e5f6b47584"
server_key_iv: "67c4dd9c8cb7aaccaeec0b78b512b853"
server_key_version: "encryptionkey@0"
I20260806 03:46:51.034590 13881 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785988004637995-12279-0/minicluster-data/ts-0/wal/instance:
uuid: "f499b5e4550248b6b408d516e5cea4c5"
format_stamp: "Formatted at 2026-08-06 03:46:51 on dist-test-slave-6ntk"
server_key: "ded0e72a62ad734bc28ca6e5f6b47584"
server_key_iv: "67c4dd9c8cb7aaccaeec0b78b512b853"
server_key_version: "encryptionkey@0"
I20260806 03:46:51.036568 13881 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260806 03:46:51.038726 13904 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:46:51.038918 13881 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 03:46:51.039024 13881 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785988004637995-12279-0/minicluster-data/ts-0/data,/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785988004637995-12279-0/minicluster-data/ts-0/wal
uuid: "f499b5e4550248b6b408d516e5cea4c5"
format_stamp: "Formatted at 2026-08-06 03:46:51 on dist-test-slave-6ntk"
server_key: "ded0e72a62ad734bc28ca6e5f6b47584"
server_key_iv: "67c4dd9c8cb7aaccaeec0b78b512b853"
server_key_version: "encryptionkey@0"
I20260806 03:46:51.039115 13881 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785988004637995-12279-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785988004637995-12279-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785988004637995-12279-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:46:51.049955 13881 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:46:51.052645 13881 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:46:51.052829 13881 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:46:51.053395 13881 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 03:46:51.053449 13881 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 03:46:51.053488 13881 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 03:46:51.053520 13881 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 03:46:51.061570 13881 rpc_server.cc:307] RPC server started. Bound to: 127.11.253.193:38889
I20260806 03:46:51.061606 14059 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.253.193:38889 every 8 connection(s)
Aug 06 03:46:51 dist-test-slave-6ntk krb5kdc[13717](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988011, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.253.193@KRBTEST.COM for kudu/127.11.253.254@KRBTEST.COM
I20260806 03:46:51.062052 13881 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785988004637995-12279-0/minicluster-data/ts-0/data/info.pb
I20260806 03:46:51.065184 12279 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskToVWRe/build/release/bin/kudu as pid 13881
I20260806 03:46:51.065300 12279 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785988004637995-12279-0/minicluster-data/ts-0/wal/instance
I20260806 03:46:51.065475 13852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:46:51.054044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.253.193:57463 (local address 127.11.253.254:37929)
0806 03:46:51.054089 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0806 03:46:51.054092 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 03:46:51.054398 (+ 306us) server_negotiation.cc:407] Connection header received
0806 03:46:51.055156 (+ 758us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:46:51.055157 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:46:51.055172 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:46:51.055244 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:46:51.056069 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:46:51.056678 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:46:51.058271 (+ 1593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:46:51.058635 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:46:51.058725 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:46:51.062387 (+ 3662us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:46:51.062388 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:46:51.062389 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:46:51.062397 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:46:51.063909 (+ 1512us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:46:51.064442 (+ 533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:46:51.064444 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:46:51.064444 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:46:51.064509 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:46:51.064774 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:46:51.064775 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:46:51.064775 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:46:51.064906 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:46:51.064979 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0806 03:46:51.065270 (+ 291us) server_negotiation.cc:299] Negotiation successful
0806 03:46:51.065378 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260806 03:46:51.065500 12279 external_mini_cluster.cc:1428] Setting key f4facd0048875961e8a68ccfdc9e5fae
I20260806 03:46:51.065662 13917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:46:51.054056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:37929 (local address 127.11.253.193:57463)
0806 03:46:51.054248 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0806 03:46:51.054297 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0806 03:46:51.055002 (+ 705us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:46:51.055327 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:46:51.055331 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:46:51.055694 (+ 363us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:46:51.055969 (+ 275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:46:51.055975 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:46:51.056778 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:46:51.056779 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:46:51.058143 (+ 1364us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:46:51.058146 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:46:51.058762 (+ 616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:46:51.058763 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:46:51.058916 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:46:51.060483 (+ 1567us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:46:51.060493 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:46:51.062268 (+ 1775us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:46:51.064017 (+ 1749us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:46:51.064025 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:46:51.064028 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:46:51.064341 (+ 313us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:46:51.064578 (+ 237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:46:51.064579 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:46:51.064580 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:46:51.064697 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:46:51.064983 (+ 286us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:46:51.064991 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:46:51.065131 (+ 140us) client_negotiation.cc:769] Sending connection context
0806 03:46:51.065237 (+ 106us) client_negotiation.cc:240] Negotiation successful
0806 03:46:51.065412 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":96,"threads_started":1}
I20260806 03:46:51.066344 14060 heartbeater.cc:347] Connected to a master server at 127.11.253.254:37929
I20260806 03:46:51.066454 14060 heartbeater.cc:464] Registering TS with master...
I20260806 03:46:51.066655 14060 heartbeater.cc:511] Master 127.11.253.254:37929 requested a full tablet report, sending...
I20260806 03:46:51.067195 13781 ts_manager.cc:194] Registered new tserver with Master: f499b5e4550248b6b408d516e5cea4c5 (127.11.253.193:38889)
I20260806 03:46:51.067869 13781 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.253.193@KRBTEST.COM'} at 127.11.253.193:57463
I20260806 03:46:51.075852 13852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:46:51.067588 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.253.193:39901 (local address 127.11.253.254:37929)
0806 03:46:51.068344 (+ 756us) server_negotiation.cc:206] Beginning negotiation
0806 03:46:51.068347 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 03:46:51.068677 (+ 330us) server_negotiation.cc:407] Connection header received
0806 03:46:51.068752 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:46:51.068753 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:46:51.068764 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:46:51.068821 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:46:51.069193 (+ 372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:46:51.069963 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:46:51.071068 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:46:51.071392 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:46:51.071444 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:46:51.073163 (+ 1719us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:46:51.073165 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:46:51.073165 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:46:51.073173 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:46:51.074706 (+ 1533us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:46:51.075074 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:46:51.075075 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:46:51.075076 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:46:51.075135 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:46:51.075324 (+ 189us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:46:51.075325 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:46:51.075325 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:46:51.075441 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:46:51.075521 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0806 03:46:51.075656 (+ 135us) server_negotiation.cc:299] Negotiation successful
0806 03:46:51.075746 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":734}
I20260806 03:46:51.075871 14069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:46:51.068435 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:37929 (local address 127.11.253.193:39901)
0806 03:46:51.068594 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0806 03:46:51.068605 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 03:46:51.068693 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:46:51.068826 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:46:51.068826 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:46:51.069061 (+ 235us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:46:51.069115 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:46:51.069117 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:46:51.070108 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:46:51.070109 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:46:51.070969 (+ 860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:46:51.070973 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:46:51.071528 (+ 555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:46:51.071529 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:46:51.071613 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:46:51.072642 (+ 1029us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:46:51.072647 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:46:51.073064 (+ 417us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:46:51.074824 (+ 1760us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:46:51.074829 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:46:51.074830 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:46:51.074995 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:46:51.075208 (+ 213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:46:51.075208 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:46:51.075209 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:46:51.075268 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:46:51.075512 (+ 244us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:46:51.075513 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:46:51.075561 (+ 48us) client_negotiation.cc:769] Sending connection context
0806 03:46:51.075630 (+ 69us) client_negotiation.cc:240] Negotiation successful
0806 03:46:51.075749 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":89,"threads_started":1}
WARNING: no policy specified for kudu/127.11.253.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.253.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.253.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.194.keytab.
Entry for principal kudu/127.11.253.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.194.keytab.
I20260806 03:46:51.097203 12279 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskToVWRe/build/release/bin/kudu
/tmp/dist-test-taskToVWRe/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785988004637995-12279-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785988004637995-12279-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785988004637995-12279-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785988004637995-12279-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.253.194:0
--local_ip_for_outbound_sockets=127.11.253.194
--webserver_interface=127.11.253.194
--webserver_port=0
--tserver_master_addrs=127.11.253.254:37929
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.194.keytab
--principal=kudu/127.11.253.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.253.212:39969
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785988004637995-12279-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785988004637995-12279-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785988004637995-12279-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:46:51.182308 14071 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:46:51.182543 14071 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:46:51.182569 14071 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:46:51.183794 14071 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:46:51.183840 14071 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:46:51.183861 14071 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 03:46:51.183912 14071 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.253.194
I20260806 03:46:51.185509 14071 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.253.212:39969
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785988004637995-12279-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785988004637995-12279-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.194.keytab
--principal=kudu/127.11.253.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.253.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785988004637995-12279-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.253.194
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.11.253.254:37929
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14071
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.253.194
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785988004637995-12279-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:46:51.185772 14071 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:46:51.186060 14071 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:46:51.191386 14079 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:46:51.191445 14071 server_base.cc:1034] running on GCE node
W20260806 03:46:51.191380 14078 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:46:51.191386 14082 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:46:51.191816 14071 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:46:51.192085 14071 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:46:51.193279 14071 hybrid_clock.cc:648] HybridClock initialized: now 1785988011193265 us; error 38 us; skew 500 ppm
Aug 06 03:46:51 dist-test-slave-6ntk krb5kdc[13717](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988011, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.253.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:46:51.196092 14071 init.cc:377] Logged in from keytab as kudu/127.11.253.194@KRBTEST.COM (short username kudu)
I20260806 03:46:51.196560 14071 webserver.cc:460] Webserver started at http://127.11.253.194:44465/ using document root <none> and password file <none>
I20260806 03:46:51.196825 14071 fs_manager.cc:359] Metadata directory not provided
I20260806 03:46:51.196900 14071 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:46:51.197055 14071 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:46:51.198189 14071 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785988004637995-12279-0/minicluster-data/ts-1/data/instance:
uuid: "7b056c9cca154ce48cc5646e1df16173"
format_stamp: "Formatted at 2026-08-06 03:46:51 on dist-test-slave-6ntk"
server_key: "2b8a03dc2443b462dd7583c48625853b"
server_key_iv: "057818a23e3651308572ce68f0166ca2"
server_key_version: "encryptionkey@0"
I20260806 03:46:51.198562 14071 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785988004637995-12279-0/minicluster-data/ts-1/wal/instance:
uuid: "7b056c9cca154ce48cc5646e1df16173"
format_stamp: "Formatted at 2026-08-06 03:46:51 on dist-test-slave-6ntk"
server_key: "2b8a03dc2443b462dd7583c48625853b"
server_key_iv: "057818a23e3651308572ce68f0166ca2"
server_key_version: "encryptionkey@0"
I20260806 03:46:51.199949 14071 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 03:46:51.200783 14091 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:46:51.200958 14071 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 03:46:51.201071 14071 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785988004637995-12279-0/minicluster-data/ts-1/data,/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785988004637995-12279-0/minicluster-data/ts-1/wal
uuid: "7b056c9cca154ce48cc5646e1df16173"
format_stamp: "Formatted at 2026-08-06 03:46:51 on dist-test-slave-6ntk"
server_key: "2b8a03dc2443b462dd7583c48625853b"
server_key_iv: "057818a23e3651308572ce68f0166ca2"
server_key_version: "encryptionkey@0"
I20260806 03:46:51.201148 14071 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785988004637995-12279-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785988004637995-12279-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785988004637995-12279-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:46:51.213428 14071 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:46:51.215574 14071 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:46:51.215742 14071 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:46:51.216266 14071 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 03:46:51.216333 14071 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 03:46:51.216406 14071 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 03:46:51.216435 14071 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 03:46:51 dist-test-slave-6ntk krb5kdc[13717](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988011, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.253.194@KRBTEST.COM for kudu/127.11.253.254@KRBTEST.COM
I20260806 03:46:51.223793 14071 rpc_server.cc:307] RPC server started. Bound to: 127.11.253.194:45365
I20260806 03:46:51.223873 14240 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.253.194:45365 every 8 connection(s)
I20260806 03:46:51.224210 14071 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785988004637995-12279-0/minicluster-data/ts-1/data/info.pb
I20260806 03:46:51.226744 13852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:46:51.216864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.253.194:57243 (local address 127.11.253.254:37929)
0806 03:46:51.216976 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0806 03:46:51.216978 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:46:51.217134 (+ 156us) server_negotiation.cc:407] Connection header received
0806 03:46:51.217616 (+ 482us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:46:51.217618 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:46:51.217632 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:46:51.217712 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:46:51.218521 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:46:51.219375 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:46:51.220985 (+ 1610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:46:51.221343 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:46:51.221457 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:46:51.223874 (+ 2417us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:46:51.223875 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:46:51.223875 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:46:51.223883 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:46:51.225474 (+ 1591us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:46:51.225841 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:46:51.225842 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:46:51.225842 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:46:51.225954 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:46:51.226173 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:46:51.226174 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:46:51.226174 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:46:51.226310 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:46:51.226365 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0806 03:46:51.226613 (+ 248us) server_negotiation.cc:299] Negotiation successful
0806 03:46:51.226678 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260806 03:46:51.226884 14106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:46:51.216819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:37929 (local address 127.11.253.194:57243)
0806 03:46:51.216998 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0806 03:46:51.217041 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 03:46:51.217520 (+ 479us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:46:51.217828 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:46:51.217832 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:46:51.218204 (+ 372us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:46:51.218412 (+ 208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:46:51.218417 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:46:51.219479 (+ 1062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:46:51.219480 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:46:51.220876 (+ 1396us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:46:51.220878 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:46:51.221475 (+ 597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:46:51.221476 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:46:51.221654 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:46:51.222243 (+ 589us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:46:51.222252 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:46:51.223757 (+ 1505us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:46:51.225550 (+ 1793us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:46:51.225554 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:46:51.225556 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:46:51.225770 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:46:51.226018 (+ 248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:46:51.226018 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:46:51.226019 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:46:51.226116 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:46:51.226378 (+ 262us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:46:51.226383 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:46:51.226495 (+ 112us) client_negotiation.cc:769] Sending connection context
0806 03:46:51.226578 (+ 83us) client_negotiation.cc:240] Negotiation successful
0806 03:46:51.226695 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":85,"threads_started":1}
I20260806 03:46:51.227362 14242 heartbeater.cc:347] Connected to a master server at 127.11.253.254:37929
I20260806 03:46:51.227481 14242 heartbeater.cc:464] Registering TS with master...
I20260806 03:46:51.227674 14242 heartbeater.cc:511] Master 127.11.253.254:37929 requested a full tablet report, sending...
I20260806 03:46:51.228030 13783 ts_manager.cc:194] Registered new tserver with Master: 7b056c9cca154ce48cc5646e1df16173 (127.11.253.194:45365)
I20260806 03:46:51.228477 13783 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.253.194@KRBTEST.COM'} at 127.11.253.194:57243
I20260806 03:46:51.233153 12279 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskToVWRe/build/release/bin/kudu as pid 14071
I20260806 03:46:51.233263 12279 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785988004637995-12279-0/minicluster-data/ts-1/wal/instance
I20260806 03:46:51.233453 12279 external_mini_cluster.cc:1428] Setting key 01a029f60e699e48f75fa9eeac0faf11
I20260806 03:46:51.233780 13852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:46:51.228474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.253.194:45805 (local address 127.11.253.254:37929)
0806 03:46:51.228513 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0806 03:46:51.228515 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:46:51.228723 (+ 208us) server_negotiation.cc:407] Connection header received
0806 03:46:51.228800 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:46:51.228801 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:46:51.228811 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:46:51.228860 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:46:51.229170 (+ 310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:46:51.229693 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:46:51.230450 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:46:51.230670 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:46:51.230727 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:46:51.231510 (+ 783us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:46:51.231511 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:46:51.231511 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:46:51.231519 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:46:51.232608 (+ 1089us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:46:51.232940 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:46:51.232941 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:46:51.232941 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:46:51.232994 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:46:51.233193 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:46:51.233194 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:46:51.233195 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:46:51.233383 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:46:51.233449 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0806 03:46:51.233622 (+ 173us) server_negotiation.cc:299] Negotiation successful
0806 03:46:51.233695 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260806 03:46:51.233822 14250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:46:51.228442 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:37929 (local address 127.11.253.194:45805)
0806 03:46:51.228641 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0806 03:46:51.228652 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 03:46:51.228736 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:46:51.228864 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:46:51.228865 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:46:51.229040 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:46:51.229095 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:46:51.229096 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:46:51.229790 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:46:51.229791 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:46:51.230360 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:46:51.230362 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:46:51.230726 (+ 364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:46:51.230727 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:46:51.230808 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:46:51.231163 (+ 355us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:46:51.231167 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:46:51.231424 (+ 257us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:46:51.232690 (+ 1266us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:46:51.232691 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:46:51.232692 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:46:51.232868 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:46:51.233066 (+ 198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:46:51.233067 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:46:51.233067 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:46:51.233125 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:46:51.233449 (+ 324us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:46:51.233450 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:46:51.233497 (+ 47us) client_negotiation.cc:769] Sending connection context
0806 03:46:51.233588 (+ 91us) client_negotiation.cc:240] Negotiation successful
0806 03:46:51.233692 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":129,"threads_started":1}
WARNING: no policy specified for kudu/127.11.253.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.253.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.253.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.195.keytab.
Entry for principal kudu/127.11.253.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.195.keytab.
I20260806 03:46:51.261610 12279 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskToVWRe/build/release/bin/kudu
/tmp/dist-test-taskToVWRe/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785988004637995-12279-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785988004637995-12279-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785988004637995-12279-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785988004637995-12279-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.253.195:0
--local_ip_for_outbound_sockets=127.11.253.195
--webserver_interface=127.11.253.195
--webserver_port=0
--tserver_master_addrs=127.11.253.254:37929
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.195.keytab
--principal=kudu/127.11.253.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.253.212:39969
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785988004637995-12279-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785988004637995-12279-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785988004637995-12279-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:46:51.347321 14254 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:46:51.347554 14254 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:46:51.347579 14254 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:46:51.348773 14254 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:46:51.348819 14254 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:46:51.348839 14254 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 03:46:51.348856 14254 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.253.195
I20260806 03:46:51.350322 14254 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.253.212:39969
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785988004637995-12279-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785988004637995-12279-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.195.keytab
--principal=kudu/127.11.253.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.253.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785988004637995-12279-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.253.195
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.11.253.254:37929
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14254
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.253.195
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785988004637995-12279-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:46:51.350577 14254 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:46:51.350826 14254 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:46:51.355989 14263 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:46:51.355958 14260 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:46:51.356003 14259 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:46:51.356078 14254 server_base.cc:1034] running on GCE node
I20260806 03:46:51.356403 14254 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:46:51.356662 14254 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:46:51.357836 14254 hybrid_clock.cc:648] HybridClock initialized: now 1785988011357835 us; error 48 us; skew 500 ppm
Aug 06 03:46:51 dist-test-slave-6ntk krb5kdc[13717](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988011, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.253.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:46:51.360451 14254 init.cc:377] Logged in from keytab as kudu/127.11.253.195@KRBTEST.COM (short username kudu)
I20260806 03:46:51.361023 14254 webserver.cc:460] Webserver started at http://127.11.253.195:43547/ using document root <none> and password file <none>
I20260806 03:46:51.361284 14254 fs_manager.cc:359] Metadata directory not provided
I20260806 03:46:51.361366 14254 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:46:51.361521 14254 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:46:51.362643 14254 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785988004637995-12279-0/minicluster-data/ts-2/data/instance:
uuid: "15d976393f624eb4b5f44f848a656b92"
format_stamp: "Formatted at 2026-08-06 03:46:51 on dist-test-slave-6ntk"
server_key: "c58be54205de0f05da1524a72dd8e17f"
server_key_iv: "202fd2e8ed4664e3e0b17c06a72cb55e"
server_key_version: "encryptionkey@0"
I20260806 03:46:51.363010 14254 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785988004637995-12279-0/minicluster-data/ts-2/wal/instance:
uuid: "15d976393f624eb4b5f44f848a656b92"
format_stamp: "Formatted at 2026-08-06 03:46:51 on dist-test-slave-6ntk"
server_key: "c58be54205de0f05da1524a72dd8e17f"
server_key_iv: "202fd2e8ed4664e3e0b17c06a72cb55e"
server_key_version: "encryptionkey@0"
I20260806 03:46:51.364391 14254 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260806 03:46:51.365139 14275 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:46:51.365340 14254 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 03:46:51.365456 14254 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785988004637995-12279-0/minicluster-data/ts-2/data,/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785988004637995-12279-0/minicluster-data/ts-2/wal
uuid: "15d976393f624eb4b5f44f848a656b92"
format_stamp: "Formatted at 2026-08-06 03:46:51 on dist-test-slave-6ntk"
server_key: "c58be54205de0f05da1524a72dd8e17f"
server_key_iv: "202fd2e8ed4664e3e0b17c06a72cb55e"
server_key_version: "encryptionkey@0"
I20260806 03:46:51.365532 14254 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785988004637995-12279-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785988004637995-12279-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785988004637995-12279-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:46:51.384881 14254 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:46:51.387306 14254 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:46:51.387486 14254 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:46:51.388123 14254 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 03:46:51.388175 14254 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 03:46:51.388212 14254 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 03:46:51.388264 14254 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 03:46:51 dist-test-slave-6ntk krb5kdc[13717](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988011, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.253.195@KRBTEST.COM for kudu/127.11.253.254@KRBTEST.COM
I20260806 03:46:51.395977 14254 rpc_server.cc:307] RPC server started. Bound to: 127.11.253.195:39711
I20260806 03:46:51.396142 14433 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.253.195:39711 every 8 connection(s)
I20260806 03:46:51.396376 14254 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785988004637995-12279-0/minicluster-data/ts-2/data/info.pb
I20260806 03:46:51.397367 12279 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskToVWRe/build/release/bin/kudu as pid 14254
I20260806 03:46:51.397455 12279 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785988004637995-12279-0/minicluster-data/ts-2/wal/instance
I20260806 03:46:51.397629 12279 external_mini_cluster.cc:1428] Setting key efa1cf682ff4252ff03f0e8d07f2cb55
I20260806 03:46:51.398977 13852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:46:51.388569 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.253.195:44277 (local address 127.11.253.254:37929)
0806 03:46:51.388642 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0806 03:46:51.388645 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 03:46:51.388901 (+ 256us) server_negotiation.cc:407] Connection header received
0806 03:46:51.389385 (+ 484us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:46:51.389387 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:46:51.389400 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:46:51.389489 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:46:51.390160 (+ 671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:46:51.390684 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:46:51.391801 (+ 1117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:46:51.392017 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:46:51.392137 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:46:51.395596 (+ 3459us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:46:51.395597 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:46:51.395597 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:46:51.395605 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:46:51.397326 (+ 1721us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:46:51.397861 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:46:51.397863 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:46:51.397863 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:46:51.397952 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:46:51.398321 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:46:51.398323 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:46:51.398323 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:46:51.398480 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:46:51.398556 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0806 03:46:51.398838 (+ 282us) server_negotiation.cc:299] Negotiation successful
0806 03:46:51.398912 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260806 03:46:51.399113 14288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:46:51.388564 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:37929 (local address 127.11.253.195:44277)
0806 03:46:51.388754 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0806 03:46:51.388797 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 03:46:51.389288 (+ 491us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:46:51.389548 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:46:51.389551 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:46:51.389784 (+ 233us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:46:51.390058 (+ 274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:46:51.390063 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:46:51.390789 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:46:51.390790 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:46:51.391708 (+ 918us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:46:51.391710 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:46:51.392133 (+ 423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:46:51.392134 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:46:51.392276 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:46:51.392967 (+ 691us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:46:51.392977 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:46:51.395487 (+ 2510us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:46:51.397415 (+ 1928us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:46:51.397421 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:46:51.397423 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:46:51.397770 (+ 347us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:46:51.398050 (+ 280us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:46:51.398051 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:46:51.398052 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:46:51.398193 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:46:51.398565 (+ 372us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:46:51.398572 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:46:51.398716 (+ 144us) client_negotiation.cc:769] Sending connection context
0806 03:46:51.398797 (+ 81us) client_negotiation.cc:240] Negotiation successful
0806 03:46:51.398921 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":114,"threads_started":1}
I20260806 03:46:51.399614 14434 heartbeater.cc:347] Connected to a master server at 127.11.253.254:37929
I20260806 03:46:51.399713 14434 heartbeater.cc:464] Registering TS with master...
I20260806 03:46:51.399888 14434 heartbeater.cc:511] Master 127.11.253.254:37929 requested a full tablet report, sending...
I20260806 03:46:51.400255 13783 ts_manager.cc:194] Registered new tserver with Master: 15d976393f624eb4b5f44f848a656b92 (127.11.253.195:39711)
I20260806 03:46:51.400744 13783 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.253.195@KRBTEST.COM'} at 127.11.253.195:44277
I20260806 03:46:51.406066 13852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:46:51.400676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.253.195:39779 (local address 127.11.253.254:37929)
0806 03:46:51.400719 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0806 03:46:51.400721 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:46:51.400853 (+ 132us) server_negotiation.cc:407] Connection header received
0806 03:46:51.400930 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:46:51.400931 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:46:51.400941 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:46:51.400986 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:46:51.401381 (+ 395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:46:51.401851 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:46:51.402597 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:46:51.402811 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:46:51.402883 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:46:51.403809 (+ 926us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:46:51.403810 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:46:51.403811 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:46:51.403817 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:46:51.405085 (+ 1268us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:46:51.405474 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:46:51.405475 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:46:51.405475 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:46:51.405525 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:46:51.405675 (+ 150us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:46:51.405676 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:46:51.405676 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:46:51.405759 (+ 83us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:46:51.405803 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0806 03:46:51.405948 (+ 145us) server_negotiation.cc:299] Negotiation successful
0806 03:46:51.406007 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260806 03:46:51.406085 14443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:46:51.400620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:37929 (local address 127.11.253.195:39779)
0806 03:46:51.400783 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0806 03:46:51.400793 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 03:46:51.400868 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:46:51.400998 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:46:51.400999 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:46:51.401224 (+ 225us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:46:51.401284 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:46:51.401286 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:46:51.401969 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:46:51.401970 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:46:51.402499 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:46:51.402500 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:46:51.402882 (+ 382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:46:51.402883 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:46:51.402961 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:46:51.403409 (+ 448us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:46:51.403414 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:46:51.403692 (+ 278us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:46:51.405155 (+ 1463us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:46:51.405155 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:46:51.405156 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:46:51.405408 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:46:51.405585 (+ 177us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:46:51.405586 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:46:51.405586 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:46:51.405638 (+ 52us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:46:51.405794 (+ 156us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:46:51.405795 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:46:51.405828 (+ 33us) client_negotiation.cc:769] Sending connection context
0806 03:46:51.405915 (+ 87us) client_negotiation.cc:240] Negotiation successful
0806 03:46:51.406023 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":71,"threads_started":1}
I20260806 03:46:51.409518 12279 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 06 03:46:51 dist-test-slave-6ntk krb5kdc[13717](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988011, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 06 03:46:51 dist-test-slave-6ntk krb5kdc[13717](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988011, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.11.253.254@KRBTEST.COM
I20260806 03:46:51.433600 13852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:46:51.426564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52534 (local address 127.11.253.254:37929)
0806 03:46:51.426627 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0806 03:46:51.426629 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:46:51.426639 (+ 10us) server_negotiation.cc:407] Connection header received
0806 03:46:51.426672 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:46:51.426673 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:46:51.426681 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:46:51.426731 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:46:51.427332 (+ 601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:46:51.427835 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:46:51.428654 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:46:51.428832 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:46:51.428888 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:46:51.431318 (+ 2430us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:46:51.431319 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:46:51.431320 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:46:51.431327 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:46:51.432546 (+ 1219us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:46:51.432869 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:46:51.432870 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:46:51.432870 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:46:51.432937 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:46:51.433125 (+ 188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:46:51.433126 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:46:51.433126 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:46:51.433225 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:46:51.433301 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0806 03:46:51.433475 (+ 174us) server_negotiation.cc:299] Negotiation successful
0806 03:46:51.433533 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260806 03:46:51.435642 13783 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52534:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 03:46:51.437275 13783 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 03:46:51.444495 14456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:46:51.440275 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59570 (local address 127.11.253.194:45365)
0806 03:46:51.440459 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0806 03:46:51.440465 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 03:46:51.440691 (+ 226us) server_negotiation.cc:407] Connection header received
0806 03:46:51.440849 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:46:51.440854 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:46:51.440898 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:46:51.440959 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 03:46:51.441341 (+ 382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:46:51.442107 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:46:51.443400 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:46:51.444051 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:46:51.444139 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:46:51.444200 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0806 03:46:51.444361 (+ 161us) server_negotiation.cc:299] Negotiation successful
0806 03:46:51.444398 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":115,"threads_started":1}
I20260806 03:46:51.444487 14457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:46:51.440120 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.194:45365 (local address 127.0.0.1:59570)
0806 03:46:51.440566 (+ 446us) negotiation.cc:107] Waiting for socket to connect
0806 03:46:51.440611 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 03:46:51.440741 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:46:51.441078 (+ 337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:46:51.441078 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:46:51.441081 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 03:46:51.441230 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:46:51.441232 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:46:51.442189 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:46:51.442189 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:46:51.443271 (+ 1082us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:46:51.443274 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:46:51.444136 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:46:51.444138 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:46:51.444218 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:46:51.444231 (+ 13us) client_negotiation.cc:769] Sending connection context
0806 03:46:51.444340 (+ 109us) client_negotiation.cc:240] Negotiation successful
0806 03:46:51.444398 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":410,"mutex_wait_us":37,"thread_start_us":81,"threads_started":1}
I20260806 03:46:51.445082 14454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:46:51.440415 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.193:38889 (local address 127.0.0.1:53754)
0806 03:46:51.440812 (+ 397us) negotiation.cc:107] Waiting for socket to connect
0806 03:46:51.440821 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 03:46:51.440885 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:46:51.441072 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:46:51.441076 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:46:51.441078 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 03:46:51.441192 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:46:51.441195 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:46:51.442641 (+ 1446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:46:51.442642 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:46:51.443897 (+ 1255us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:46:51.443900 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:46:51.444837 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:46:51.444838 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:46:51.444901 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:46:51.444906 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 03:46:51.444943 (+ 37us) client_negotiation.cc:240] Negotiation successful
0806 03:46:51.444996 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":388,"mutex_wait_us":88,"thread_start_us":86,"threads_started":1}
I20260806 03:46:51.445102 14455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:46:51.440118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.195:39711 (local address 127.0.0.1:40998)
0806 03:46:51.440567 (+ 449us) negotiation.cc:107] Waiting for socket to connect
0806 03:46:51.440609 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 03:46:51.440700 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:46:51.441226 (+ 526us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:46:51.441227 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:46:51.441228 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 03:46:51.441269 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:46:51.441269 (+ 0us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:46:51.442641 (+ 1372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:46:51.442642 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:46:51.443897 (+ 1255us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:46:51.443900 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:46:51.444818 (+ 918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:46:51.444819 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:46:51.444885 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:46:51.444890 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 03:46:51.444945 (+ 55us) client_negotiation.cc:240] Negotiation successful
0806 03:46:51.445001 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":419,"mutex_wait_us":70,"thread_start_us":67,"threads_started":1}
I20260806 03:46:51.445441 14458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:46:51.440403 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40998 (local address 127.11.253.195:39711)
0806 03:46:51.441041 (+ 638us) server_negotiation.cc:206] Beginning negotiation
0806 03:46:51.441045 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 03:46:51.441055 (+ 10us) server_negotiation.cc:407] Connection header received
0806 03:46:51.441095 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:46:51.441097 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:46:51.441113 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:46:51.441206 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 03:46:51.441350 (+ 144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:46:51.442538 (+ 1188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:46:51.444015 (+ 1477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:46:51.444729 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:46:51.444814 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:46:51.444883 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0806 03:46:51.445298 (+ 415us) server_negotiation.cc:299] Negotiation successful
0806 03:46:51.445347 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":582,"thread_start_us":98,"threads_started":1}
I20260806 03:46:51.445498 14459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:46:51.440575 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53754 (local address 127.11.253.193:38889)
0806 03:46:51.440815 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0806 03:46:51.440820 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 03:46:51.440901 (+ 81us) server_negotiation.cc:407] Connection header received
0806 03:46:51.440944 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:46:51.440947 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:46:51.440961 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:46:51.441040 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 03:46:51.441281 (+ 241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:46:51.442519 (+ 1238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:46:51.444000 (+ 1481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:46:51.444729 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:46:51.445259 (+ 530us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:46:51.445322 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0806 03:46:51.445363 (+ 41us) server_negotiation.cc:299] Negotiation successful
0806 03:46:51.445401 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":185,"thread_start_us":91,"threads_started":1}
I20260806 03:46:51.445641 14159 tablet_service.cc:1467] Processing CreateTablet for tablet eb9ab524d77c4086bd564c89aa7519f9 (DEFAULT_TABLE table=test-table [id=4e212ac184de403d9adeb06c2284f96b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 03:46:51.446105 14159 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eb9ab524d77c4086bd564c89aa7519f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:46:51.446144 13972 tablet_service.cc:1467] Processing CreateTablet for tablet eb9ab524d77c4086bd564c89aa7519f9 (DEFAULT_TABLE table=test-table [id=4e212ac184de403d9adeb06c2284f96b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 03:46:51.446126 14343 tablet_service.cc:1467] Processing CreateTablet for tablet eb9ab524d77c4086bd564c89aa7519f9 (DEFAULT_TABLE table=test-table [id=4e212ac184de403d9adeb06c2284f96b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 03:46:51.446494 13972 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eb9ab524d77c4086bd564c89aa7519f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:46:51.446494 14343 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eb9ab524d77c4086bd564c89aa7519f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:46:51.449028 14460 tablet_bootstrap.cc:492] T eb9ab524d77c4086bd564c89aa7519f9 P 7b056c9cca154ce48cc5646e1df16173: Bootstrap starting.
I20260806 03:46:51.449726 14460 tablet_bootstrap.cc:654] T eb9ab524d77c4086bd564c89aa7519f9 P 7b056c9cca154ce48cc5646e1df16173: Neither blocks nor log segments found. Creating new log.
I20260806 03:46:51.449717 14461 tablet_bootstrap.cc:492] T eb9ab524d77c4086bd564c89aa7519f9 P 15d976393f624eb4b5f44f848a656b92: Bootstrap starting.
I20260806 03:46:51.450232 14460 log.cc:824] T eb9ab524d77c4086bd564c89aa7519f9 P 7b056c9cca154ce48cc5646e1df16173: Log is configured to *not* fsync() on all Append() calls
I20260806 03:46:51.450609 14462 tablet_bootstrap.cc:492] T eb9ab524d77c4086bd564c89aa7519f9 P f499b5e4550248b6b408d516e5cea4c5: Bootstrap starting.
I20260806 03:46:51.450645 14461 tablet_bootstrap.cc:654] T eb9ab524d77c4086bd564c89aa7519f9 P 15d976393f624eb4b5f44f848a656b92: Neither blocks nor log segments found. Creating new log.
I20260806 03:46:51.450989 14461 log.cc:824] T eb9ab524d77c4086bd564c89aa7519f9 P 15d976393f624eb4b5f44f848a656b92: Log is configured to *not* fsync() on all Append() calls
I20260806 03:46:51.451050 14460 tablet_bootstrap.cc:492] T eb9ab524d77c4086bd564c89aa7519f9 P 7b056c9cca154ce48cc5646e1df16173: No bootstrap required, opened a new log
I20260806 03:46:51.451108 14460 ts_tablet_manager.cc:1397] T eb9ab524d77c4086bd564c89aa7519f9 P 7b056c9cca154ce48cc5646e1df16173: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 03:46:51.451742 14461 tablet_bootstrap.cc:492] T eb9ab524d77c4086bd564c89aa7519f9 P 15d976393f624eb4b5f44f848a656b92: No bootstrap required, opened a new log
I20260806 03:46:51.451819 14461 ts_tablet_manager.cc:1397] T eb9ab524d77c4086bd564c89aa7519f9 P 15d976393f624eb4b5f44f848a656b92: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 03:46:51.453141 14460 raft_consensus.cc:348] T eb9ab524d77c4086bd564c89aa7519f9 P 7b056c9cca154ce48cc5646e1df16173 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b056c9cca154ce48cc5646e1df16173" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 45365 } } peers { permanent_uuid: "15d976393f624eb4b5f44f848a656b92" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 39711 } } peers { permanent_uuid: "f499b5e4550248b6b408d516e5cea4c5" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 38889 } }
I20260806 03:46:51.453315 14460 raft_consensus.cc:374] T eb9ab524d77c4086bd564c89aa7519f9 P 7b056c9cca154ce48cc5646e1df16173 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:46:51.453362 14460 raft_consensus.cc:729] T eb9ab524d77c4086bd564c89aa7519f9 P 7b056c9cca154ce48cc5646e1df16173 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7b056c9cca154ce48cc5646e1df16173, State: Initialized, Role: FOLLOWER
I20260806 03:46:51.453485 14460 consensus_queue.cc:260] T eb9ab524d77c4086bd564c89aa7519f9 P 7b056c9cca154ce48cc5646e1df16173 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b056c9cca154ce48cc5646e1df16173" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 45365 } } peers { permanent_uuid: "15d976393f624eb4b5f44f848a656b92" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 39711 } } peers { permanent_uuid: "f499b5e4550248b6b408d516e5cea4c5" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 38889 } }
I20260806 03:46:51.453763 14460 ts_tablet_manager.cc:1428] T eb9ab524d77c4086bd564c89aa7519f9 P 7b056c9cca154ce48cc5646e1df16173: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 03:46:51.453814 14242 heartbeater.cc:503] Master 127.11.253.254:37929 was elected leader, sending a full tablet report...
I20260806 03:46:51.453816 14461 raft_consensus.cc:348] T eb9ab524d77c4086bd564c89aa7519f9 P 15d976393f624eb4b5f44f848a656b92 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b056c9cca154ce48cc5646e1df16173" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 45365 } } peers { permanent_uuid: "15d976393f624eb4b5f44f848a656b92" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 39711 } } peers { permanent_uuid: "f499b5e4550248b6b408d516e5cea4c5" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 38889 } }
I20260806 03:46:51.454039 14461 raft_consensus.cc:374] T eb9ab524d77c4086bd564c89aa7519f9 P 15d976393f624eb4b5f44f848a656b92 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:46:51.454097 14461 raft_consensus.cc:729] T eb9ab524d77c4086bd564c89aa7519f9 P 15d976393f624eb4b5f44f848a656b92 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 15d976393f624eb4b5f44f848a656b92, State: Initialized, Role: FOLLOWER
I20260806 03:46:51.454226 14461 consensus_queue.cc:260] T eb9ab524d77c4086bd564c89aa7519f9 P 15d976393f624eb4b5f44f848a656b92 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b056c9cca154ce48cc5646e1df16173" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 45365 } } peers { permanent_uuid: "15d976393f624eb4b5f44f848a656b92" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 39711 } } peers { permanent_uuid: "f499b5e4550248b6b408d516e5cea4c5" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 38889 } }
I20260806 03:46:51.454464 14462 tablet_bootstrap.cc:654] T eb9ab524d77c4086bd564c89aa7519f9 P f499b5e4550248b6b408d516e5cea4c5: Neither blocks nor log segments found. Creating new log.
I20260806 03:46:51.454473 14461 ts_tablet_manager.cc:1428] T eb9ab524d77c4086bd564c89aa7519f9 P 15d976393f624eb4b5f44f848a656b92: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 03:46:51.454514 14434 heartbeater.cc:503] Master 127.11.253.254:37929 was elected leader, sending a full tablet report...
I20260806 03:46:51.454828 14462 log.cc:824] T eb9ab524d77c4086bd564c89aa7519f9 P f499b5e4550248b6b408d516e5cea4c5: Log is configured to *not* fsync() on all Append() calls
I20260806 03:46:51.455679 14462 tablet_bootstrap.cc:492] T eb9ab524d77c4086bd564c89aa7519f9 P f499b5e4550248b6b408d516e5cea4c5: No bootstrap required, opened a new log
I20260806 03:46:51.455754 14462 ts_tablet_manager.cc:1397] T eb9ab524d77c4086bd564c89aa7519f9 P f499b5e4550248b6b408d516e5cea4c5: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20260806 03:46:51.457230 14462 raft_consensus.cc:348] T eb9ab524d77c4086bd564c89aa7519f9 P f499b5e4550248b6b408d516e5cea4c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b056c9cca154ce48cc5646e1df16173" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 45365 } } peers { permanent_uuid: "15d976393f624eb4b5f44f848a656b92" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 39711 } } peers { permanent_uuid: "f499b5e4550248b6b408d516e5cea4c5" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 38889 } }
I20260806 03:46:51.457360 14462 raft_consensus.cc:374] T eb9ab524d77c4086bd564c89aa7519f9 P f499b5e4550248b6b408d516e5cea4c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:46:51.457382 14462 raft_consensus.cc:729] T eb9ab524d77c4086bd564c89aa7519f9 P f499b5e4550248b6b408d516e5cea4c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f499b5e4550248b6b408d516e5cea4c5, State: Initialized, Role: FOLLOWER
I20260806 03:46:51.457518 14462 consensus_queue.cc:260] T eb9ab524d77c4086bd564c89aa7519f9 P f499b5e4550248b6b408d516e5cea4c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b056c9cca154ce48cc5646e1df16173" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 45365 } } peers { permanent_uuid: "15d976393f624eb4b5f44f848a656b92" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 39711 } } peers { permanent_uuid: "f499b5e4550248b6b408d516e5cea4c5" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 38889 } }
I20260806 03:46:51.457789 14462 ts_tablet_manager.cc:1428] T eb9ab524d77c4086bd564c89aa7519f9 P f499b5e4550248b6b408d516e5cea4c5: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 03:46:51.457818 14060 heartbeater.cc:503] Master 127.11.253.254:37929 was elected leader, sending a full tablet report...
W20260806 03:46:51.474812 14244 tablet.cc:2369] T eb9ab524d77c4086bd564c89aa7519f9 P 7b056c9cca154ce48cc5646e1df16173: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 03:46:51.539584 14466 raft_consensus.cc:482] T eb9ab524d77c4086bd564c89aa7519f9 P 7b056c9cca154ce48cc5646e1df16173 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 03:46:51.539759 14466 raft_consensus.cc:504] T eb9ab524d77c4086bd564c89aa7519f9 P 7b056c9cca154ce48cc5646e1df16173 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b056c9cca154ce48cc5646e1df16173" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 45365 } } peers { permanent_uuid: "15d976393f624eb4b5f44f848a656b92" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 39711 } } peers { permanent_uuid: "f499b5e4550248b6b408d516e5cea4c5" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 38889 } }
I20260806 03:46:51.540189 14466 leader_election.cc:288] T eb9ab524d77c4086bd564c89aa7519f9 P 7b056c9cca154ce48cc5646e1df16173 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 15d976393f624eb4b5f44f848a656b92 (127.11.253.195:39711), f499b5e4550248b6b408d516e5cea4c5 (127.11.253.193:38889)
I20260806 03:46:51.543339 14106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:46:51.540413 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.195:39711 (local address 127.11.253.194:36719)
0806 03:46:51.540484 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0806 03:46:51.540498 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 03:46:51.540550 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:46:51.540718 (+ 168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:46:51.540720 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:46:51.540722 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 03:46:51.540802 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:46:51.540804 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:46:51.541548 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:46:51.541549 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:46:51.542472 (+ 923us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:46:51.542475 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:46:51.543055 (+ 580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:46:51.543057 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:46:51.543155 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:46:51.543158 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 03:46:51.543201 (+ 43us) client_negotiation.cc:240] Negotiation successful
0806 03:46:51.543236 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46,"mutex_wait_us":3}
I20260806 03:46:51.543359 14458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:46:51.540523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.253.194:36719 (local address 127.11.253.195:39711)
0806 03:46:51.540566 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0806 03:46:51.540568 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:46:51.540577 (+ 9us) server_negotiation.cc:407] Connection header received
0806 03:46:51.540608 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:46:51.540609 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:46:51.540621 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:46:51.540696 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 03:46:51.540889 (+ 193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:46:51.541468 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:46:51.542573 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:46:51.542974 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:46:51.543031 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:46:51.543038 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0806 03:46:51.543231 (+ 193us) server_negotiation.cc:299] Negotiation successful
0806 03:46:51.543279 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 03:46:51.543454 14459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:46:51.540496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.253.194:45881 (local address 127.11.253.193:38889)
0806 03:46:51.540547 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0806 03:46:51.540549 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:46:51.540709 (+ 160us) server_negotiation.cc:407] Connection header received
0806 03:46:51.540744 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:46:51.540745 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:46:51.540755 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:46:51.540817 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 03:46:51.541090 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:46:51.541599 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:46:51.542613 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:46:51.542987 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:46:51.543082 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:46:51.543110 (+ 28us) server_negotiation.cc:1035] Waiting for connection context
0806 03:46:51.543320 (+ 210us) server_negotiation.cc:299] Negotiation successful
0806 03:46:51.543366 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260806 03:46:51.543573 14472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:46:51.540430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.193:38889 (local address 127.11.253.194:45881)
0806 03:46:51.540624 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0806 03:46:51.540635 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 03:46:51.540696 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:46:51.540867 (+ 171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:46:51.540868 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:46:51.540882 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 03:46:51.541008 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:46:51.541009 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:46:51.541687 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:46:51.541688 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:46:51.542482 (+ 794us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:46:51.542484 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:46:51.543119 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:46:51.543120 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:46:51.543222 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:46:51.543227 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 03:46:51.543285 (+ 58us) client_negotiation.cc:240] Negotiation successful
0806 03:46:51.543321 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":121,"threads_started":1}
I20260806 03:46:51.543718 14374 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eb9ab524d77c4086bd564c89aa7519f9" candidate_uuid: "7b056c9cca154ce48cc5646e1df16173" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "15d976393f624eb4b5f44f848a656b92" is_pre_election: true
I20260806 03:46:51.543877 14374 raft_consensus.cc:2454] T eb9ab524d77c4086bd564c89aa7519f9 P 15d976393f624eb4b5f44f848a656b92 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7b056c9cca154ce48cc5646e1df16173 in term 0.
I20260806 03:46:51.543972 13997 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eb9ab524d77c4086bd564c89aa7519f9" candidate_uuid: "7b056c9cca154ce48cc5646e1df16173" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f499b5e4550248b6b408d516e5cea4c5" is_pre_election: true
I20260806 03:46:51.544134 13997 raft_consensus.cc:2454] T eb9ab524d77c4086bd564c89aa7519f9 P f499b5e4550248b6b408d516e5cea4c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7b056c9cca154ce48cc5646e1df16173 in term 0.
I20260806 03:46:51.544194 14093 leader_election.cc:302] T eb9ab524d77c4086bd564c89aa7519f9 P 7b056c9cca154ce48cc5646e1df16173 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 15d976393f624eb4b5f44f848a656b92, 7b056c9cca154ce48cc5646e1df16173; no voters:
I20260806 03:46:51.544332 14466 raft_consensus.cc:2781] T eb9ab524d77c4086bd564c89aa7519f9 P 7b056c9cca154ce48cc5646e1df16173 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 03:46:51.544389 14466 raft_consensus.cc:482] T eb9ab524d77c4086bd564c89aa7519f9 P 7b056c9cca154ce48cc5646e1df16173 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 03:46:51.544413 14466 raft_consensus.cc:3037] T eb9ab524d77c4086bd564c89aa7519f9 P 7b056c9cca154ce48cc5646e1df16173 [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:46:51.545490 14466 raft_consensus.cc:504] T eb9ab524d77c4086bd564c89aa7519f9 P 7b056c9cca154ce48cc5646e1df16173 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b056c9cca154ce48cc5646e1df16173" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 45365 } } peers { permanent_uuid: "15d976393f624eb4b5f44f848a656b92" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 39711 } } peers { permanent_uuid: "f499b5e4550248b6b408d516e5cea4c5" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 38889 } }
I20260806 03:46:51.545630 14466 leader_election.cc:288] T eb9ab524d77c4086bd564c89aa7519f9 P 7b056c9cca154ce48cc5646e1df16173 [CANDIDATE]: Term 1 election: Requested vote from peers 15d976393f624eb4b5f44f848a656b92 (127.11.253.195:39711), f499b5e4550248b6b408d516e5cea4c5 (127.11.253.193:38889)
I20260806 03:46:51.545806 13997 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eb9ab524d77c4086bd564c89aa7519f9" candidate_uuid: "7b056c9cca154ce48cc5646e1df16173" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f499b5e4550248b6b408d516e5cea4c5"
I20260806 03:46:51.545934 13997 raft_consensus.cc:3037] T eb9ab524d77c4086bd564c89aa7519f9 P f499b5e4550248b6b408d516e5cea4c5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:46:51.545977 14374 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eb9ab524d77c4086bd564c89aa7519f9" candidate_uuid: "7b056c9cca154ce48cc5646e1df16173" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "15d976393f624eb4b5f44f848a656b92"
I20260806 03:46:51.546062 14374 raft_consensus.cc:3037] T eb9ab524d77c4086bd564c89aa7519f9 P 15d976393f624eb4b5f44f848a656b92 [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:46:51.546734 13997 raft_consensus.cc:2454] T eb9ab524d77c4086bd564c89aa7519f9 P f499b5e4550248b6b408d516e5cea4c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7b056c9cca154ce48cc5646e1df16173 in term 1.
I20260806 03:46:51.546747 14374 raft_consensus.cc:2454] T eb9ab524d77c4086bd564c89aa7519f9 P 15d976393f624eb4b5f44f848a656b92 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7b056c9cca154ce48cc5646e1df16173 in term 1.
I20260806 03:46:51.546907 14093 leader_election.cc:302] T eb9ab524d77c4086bd564c89aa7519f9 P 7b056c9cca154ce48cc5646e1df16173 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 15d976393f624eb4b5f44f848a656b92, 7b056c9cca154ce48cc5646e1df16173; no voters:
I20260806 03:46:51.546990 14466 raft_consensus.cc:2781] T eb9ab524d77c4086bd564c89aa7519f9 P 7b056c9cca154ce48cc5646e1df16173 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 03:46:51.547119 14466 raft_consensus.cc:686] T eb9ab524d77c4086bd564c89aa7519f9 P 7b056c9cca154ce48cc5646e1df16173 [term 1 LEADER]: Becoming Leader. State: Replica: 7b056c9cca154ce48cc5646e1df16173, State: Running, Role: LEADER
I20260806 03:46:51.547219 14466 consensus_queue.cc:237] T eb9ab524d77c4086bd564c89aa7519f9 P 7b056c9cca154ce48cc5646e1df16173 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b056c9cca154ce48cc5646e1df16173" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 45365 } } peers { permanent_uuid: "15d976393f624eb4b5f44f848a656b92" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 39711 } } peers { permanent_uuid: "f499b5e4550248b6b408d516e5cea4c5" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 38889 } }
I20260806 03:46:51.548060 13781 catalog_manager.cc:5495] T eb9ab524d77c4086bd564c89aa7519f9 P 7b056c9cca154ce48cc5646e1df16173 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7b056c9cca154ce48cc5646e1df16173 (127.11.253.194). New cstate: current_term: 1 leader_uuid: "7b056c9cca154ce48cc5646e1df16173" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b056c9cca154ce48cc5646e1df16173" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 45365 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "15d976393f624eb4b5f44f848a656b92" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 39711 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f499b5e4550248b6b408d516e5cea4c5" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 38889 } health_report { overall_health: UNKNOWN } } }
W20260806 03:46:51.562839 14061 tablet.cc:2369] T eb9ab524d77c4086bd564c89aa7519f9 P f499b5e4550248b6b408d516e5cea4c5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 03:46:51.565451 14456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:46:51.562865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59578 (local address 127.11.253.194:45365)
0806 03:46:51.562934 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0806 03:46:51.562937 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 03:46:51.562947 (+ 10us) server_negotiation.cc:407] Connection header received
0806 03:46:51.563054 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:46:51.563055 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:46:51.563080 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:46:51.563161 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 03:46:51.563313 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:46:51.563782 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:46:51.564579 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:46:51.564775 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:46:51.564853 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:46:51.565062 (+ 209us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 03:46:51.565163 (+ 101us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 03:46:51.565270 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0806 03:46:51.565323 (+ 53us) server_negotiation.cc:299] Negotiation successful
0806 03:46:51.565343 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260806 03:46:51.567516 13997 raft_consensus.cc:1261] T eb9ab524d77c4086bd564c89aa7519f9 P f499b5e4550248b6b408d516e5cea4c5 [term 1 FOLLOWER]: Refusing update from remote peer 7b056c9cca154ce48cc5646e1df16173: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 03:46:51.567852 14475 consensus_queue.cc:1035] T eb9ab524d77c4086bd564c89aa7519f9 P 7b056c9cca154ce48cc5646e1df16173 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f499b5e4550248b6b408d516e5cea4c5" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 38889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 03:46:51.570324 14374 raft_consensus.cc:1261] T eb9ab524d77c4086bd564c89aa7519f9 P 15d976393f624eb4b5f44f848a656b92 [term 1 FOLLOWER]: Refusing update from remote peer 7b056c9cca154ce48cc5646e1df16173: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 03:46:51.570600 14466 consensus_queue.cc:1035] T eb9ab524d77c4086bd564c89aa7519f9 P 7b056c9cca154ce48cc5646e1df16173 [LEADER]: Connected to new peer: Peer: permanent_uuid: "15d976393f624eb4b5f44f848a656b92" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 39711 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 03:46:51.572103 14481 mvcc.cc:205] Tried to move back new op lower bound from 7315406895377408000 to 7315406895297986560. Current Snapshot: MvccSnapshot[applied={T|T < 7315406895377408000 or (T in {7315406895377408000})}]
I20260806 03:46:51.574951 13781 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52534:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 03:46:51.575131 13781 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52534:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 03:46:51.576105 13781 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2827edda0baf420a983b2c4a3c88ddaf: Sending DeleteTablet for 3 replicas of tablet eb9ab524d77c4086bd564c89aa7519f9
I20260806 03:46:51.576627 14159 tablet_service.cc:1514] Processing DeleteTablet for tablet eb9ab524d77c4086bd564c89aa7519f9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 03:46:51 UTC) from {username='kudu', principal='kudu/127.11.253.254@KRBTEST.COM'} at 127.0.0.1:59570
I20260806 03:46:51.576627 14343 tablet_service.cc:1514] Processing DeleteTablet for tablet eb9ab524d77c4086bd564c89aa7519f9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 03:46:51 UTC) from {username='kudu', principal='kudu/127.11.253.254@KRBTEST.COM'} at 127.0.0.1:40998
I20260806 03:46:51.577044 14495 tablet_replica.cc:330] T eb9ab524d77c4086bd564c89aa7519f9 P 15d976393f624eb4b5f44f848a656b92: stopping tablet replica
I20260806 03:46:51.577149 14495 raft_consensus.cc:2229] T eb9ab524d77c4086bd564c89aa7519f9 P 15d976393f624eb4b5f44f848a656b92 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 03:46:51.577123 13781 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52534:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 03:46:51.577256 13781 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 03:46:51.577255 14495 raft_consensus.cc:2258] T eb9ab524d77c4086bd564c89aa7519f9 P 15d976393f624eb4b5f44f848a656b92 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 03:46:51.577769 14495 ts_tablet_manager.cc:1905] T eb9ab524d77c4086bd564c89aa7519f9 P 15d976393f624eb4b5f44f848a656b92: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 03:46:51.578323 13972 tablet_service.cc:1514] Processing DeleteTablet for tablet eb9ab524d77c4086bd564c89aa7519f9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 03:46:51 UTC) from {username='kudu', principal='kudu/127.11.253.254@KRBTEST.COM'} at 127.0.0.1:53754
I20260806 03:46:51.578536 14343 tablet_service.cc:1467] Processing CreateTablet for tablet a26a93f800674986b06b4ce2c27f4a4a (DEFAULT_TABLE table=test-table [id=6c4f8eaf7746417281761c6086db798d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 03:46:51.578682 14343 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a26a93f800674986b06b4ce2c27f4a4a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:46:51.578718 14159 tablet_service.cc:1467] Processing CreateTablet for tablet a26a93f800674986b06b4ce2c27f4a4a (DEFAULT_TABLE table=test-table [id=6c4f8eaf7746417281761c6086db798d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 03:46:51.578833 14159 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a26a93f800674986b06b4ce2c27f4a4a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:46:51.578960 14497 tablet_replica.cc:330] T eb9ab524d77c4086bd564c89aa7519f9 P f499b5e4550248b6b408d516e5cea4c5: stopping tablet replica
I20260806 03:46:51.579136 14497 raft_consensus.cc:2229] T eb9ab524d77c4086bd564c89aa7519f9 P f499b5e4550248b6b408d516e5cea4c5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 03:46:51.579324 14497 raft_consensus.cc:2258] T eb9ab524d77c4086bd564c89aa7519f9 P f499b5e4550248b6b408d516e5cea4c5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 03:46:51.580116 14461 tablet_bootstrap.cc:492] T a26a93f800674986b06b4ce2c27f4a4a P 15d976393f624eb4b5f44f848a656b92: Bootstrap starting.
I20260806 03:46:51.580161 14460 tablet_bootstrap.cc:492] T a26a93f800674986b06b4ce2c27f4a4a P 7b056c9cca154ce48cc5646e1df16173: Bootstrap starting.
I20260806 03:46:51.580595 13972 tablet_service.cc:1467] Processing CreateTablet for tablet a26a93f800674986b06b4ce2c27f4a4a (DEFAULT_TABLE table=test-table [id=6c4f8eaf7746417281761c6086db798d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 03:46:51.580641 14460 tablet_bootstrap.cc:654] T a26a93f800674986b06b4ce2c27f4a4a P 7b056c9cca154ce48cc5646e1df16173: Neither blocks nor log segments found. Creating new log.
I20260806 03:46:51.580641 14461 tablet_bootstrap.cc:654] T a26a93f800674986b06b4ce2c27f4a4a P 15d976393f624eb4b5f44f848a656b92: Neither blocks nor log segments found. Creating new log.
I20260806 03:46:51.580718 13972 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a26a93f800674986b06b4ce2c27f4a4a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:46:51.581612 14497 ts_tablet_manager.cc:1905] T eb9ab524d77c4086bd564c89aa7519f9 P f499b5e4550248b6b408d516e5cea4c5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 03:46:51.582247 14462 tablet_bootstrap.cc:492] T a26a93f800674986b06b4ce2c27f4a4a P f499b5e4550248b6b408d516e5cea4c5: Bootstrap starting.
I20260806 03:46:51.582469 14495 ts_tablet_manager.cc:1918] T eb9ab524d77c4086bd564c89aa7519f9 P 15d976393f624eb4b5f44f848a656b92: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 03:46:51.582571 14495 log.cc:1196] T eb9ab524d77c4086bd564c89aa7519f9 P 15d976393f624eb4b5f44f848a656b92: Deleting WAL directory at /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785988004637995-12279-0/minicluster-data/ts-2/wal/wals/eb9ab524d77c4086bd564c89aa7519f9
I20260806 03:46:51.582778 14462 tablet_bootstrap.cc:654] T a26a93f800674986b06b4ce2c27f4a4a P f499b5e4550248b6b408d516e5cea4c5: Neither blocks nor log segments found. Creating new log.
I20260806 03:46:51.582877 14495 ts_tablet_manager.cc:1939] T eb9ab524d77c4086bd564c89aa7519f9 P 15d976393f624eb4b5f44f848a656b92: Deleting consensus metadata
I20260806 03:46:51.583352 13761 catalog_manager.cc:4841] TS 15d976393f624eb4b5f44f848a656b92 (127.11.253.195:39711): tablet eb9ab524d77c4086bd564c89aa7519f9 (table test-table [id=4e212ac184de403d9adeb06c2284f96b]) successfully deleted
I20260806 03:46:51.583582 14462 tablet_bootstrap.cc:492] T a26a93f800674986b06b4ce2c27f4a4a P f499b5e4550248b6b408d516e5cea4c5: No bootstrap required, opened a new log
I20260806 03:46:51.583577 14496 tablet_replica.cc:330] T eb9ab524d77c4086bd564c89aa7519f9 P 7b056c9cca154ce48cc5646e1df16173: stopping tablet replica
I20260806 03:46:51.583623 14462 ts_tablet_manager.cc:1397] T a26a93f800674986b06b4ce2c27f4a4a P f499b5e4550248b6b408d516e5cea4c5: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260806 03:46:51.583628 14497 ts_tablet_manager.cc:1918] T eb9ab524d77c4086bd564c89aa7519f9 P f499b5e4550248b6b408d516e5cea4c5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 03:46:51.583680 14496 raft_consensus.cc:2229] T eb9ab524d77c4086bd564c89aa7519f9 P 7b056c9cca154ce48cc5646e1df16173 [term 1 LEADER]: Raft consensus shutting down.
I20260806 03:46:51.583741 14497 log.cc:1196] T eb9ab524d77c4086bd564c89aa7519f9 P f499b5e4550248b6b408d516e5cea4c5: Deleting WAL directory at /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785988004637995-12279-0/minicluster-data/ts-0/wal/wals/eb9ab524d77c4086bd564c89aa7519f9
I20260806 03:46:51.583832 14496 raft_consensus.cc:2258] T eb9ab524d77c4086bd564c89aa7519f9 P 7b056c9cca154ce48cc5646e1df16173 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 03:46:51.583807 14462 raft_consensus.cc:348] T a26a93f800674986b06b4ce2c27f4a4a P f499b5e4550248b6b408d516e5cea4c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f499b5e4550248b6b408d516e5cea4c5" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 38889 } } peers { permanent_uuid: "7b056c9cca154ce48cc5646e1df16173" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 45365 } } peers { permanent_uuid: "15d976393f624eb4b5f44f848a656b92" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 39711 } }
I20260806 03:46:51.583877 14462 raft_consensus.cc:374] T a26a93f800674986b06b4ce2c27f4a4a P f499b5e4550248b6b408d516e5cea4c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:46:51.583901 14462 raft_consensus.cc:729] T a26a93f800674986b06b4ce2c27f4a4a P f499b5e4550248b6b408d516e5cea4c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f499b5e4550248b6b408d516e5cea4c5, State: Initialized, Role: FOLLOWER
I20260806 03:46:51.583950 14462 consensus_queue.cc:260] T a26a93f800674986b06b4ce2c27f4a4a P f499b5e4550248b6b408d516e5cea4c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f499b5e4550248b6b408d516e5cea4c5" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 38889 } } peers { permanent_uuid: "7b056c9cca154ce48cc5646e1df16173" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 45365 } } peers { permanent_uuid: "15d976393f624eb4b5f44f848a656b92" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 39711 } }
I20260806 03:46:51.584120 14462 ts_tablet_manager.cc:1428] T a26a93f800674986b06b4ce2c27f4a4a P f499b5e4550248b6b408d516e5cea4c5: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260806 03:46:51.584350 14496 ts_tablet_manager.cc:1905] T eb9ab524d77c4086bd564c89aa7519f9 P 7b056c9cca154ce48cc5646e1df16173: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 03:46:51.585314 14497 ts_tablet_manager.cc:1939] T eb9ab524d77c4086bd564c89aa7519f9 P f499b5e4550248b6b408d516e5cea4c5: Deleting consensus metadata
I20260806 03:46:51.585832 13760 catalog_manager.cc:4841] TS f499b5e4550248b6b408d516e5cea4c5 (127.11.253.193:38889): tablet eb9ab524d77c4086bd564c89aa7519f9 (table test-table [id=4e212ac184de403d9adeb06c2284f96b]) successfully deleted
I20260806 03:46:51.585958 14496 ts_tablet_manager.cc:1918] T eb9ab524d77c4086bd564c89aa7519f9 P 7b056c9cca154ce48cc5646e1df16173: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 03:46:51.586031 14496 log.cc:1196] T eb9ab524d77c4086bd564c89aa7519f9 P 7b056c9cca154ce48cc5646e1df16173: Deleting WAL directory at /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785988004637995-12279-0/minicluster-data/ts-1/wal/wals/eb9ab524d77c4086bd564c89aa7519f9
I20260806 03:46:51.586270 14496 ts_tablet_manager.cc:1939] T eb9ab524d77c4086bd564c89aa7519f9 P 7b056c9cca154ce48cc5646e1df16173: Deleting consensus metadata
I20260806 03:46:51.586671 13760 catalog_manager.cc:4841] TS 7b056c9cca154ce48cc5646e1df16173 (127.11.253.194:45365): tablet eb9ab524d77c4086bd564c89aa7519f9 (table test-table [id=4e212ac184de403d9adeb06c2284f96b]) successfully deleted
I20260806 03:46:51.590372 14461 tablet_bootstrap.cc:492] T a26a93f800674986b06b4ce2c27f4a4a P 15d976393f624eb4b5f44f848a656b92: No bootstrap required, opened a new log
I20260806 03:46:51.590416 14460 tablet_bootstrap.cc:492] T a26a93f800674986b06b4ce2c27f4a4a P 7b056c9cca154ce48cc5646e1df16173: No bootstrap required, opened a new log
I20260806 03:46:51.590444 14461 ts_tablet_manager.cc:1397] T a26a93f800674986b06b4ce2c27f4a4a P 15d976393f624eb4b5f44f848a656b92: Time spent bootstrapping tablet: real 0.010s user 0.001s sys 0.000s
I20260806 03:46:51.590457 14460 ts_tablet_manager.cc:1397] T a26a93f800674986b06b4ce2c27f4a4a P 7b056c9cca154ce48cc5646e1df16173: Time spent bootstrapping tablet: real 0.010s user 0.001s sys 0.000s
I20260806 03:46:51.590615 14460 raft_consensus.cc:348] T a26a93f800674986b06b4ce2c27f4a4a P 7b056c9cca154ce48cc5646e1df16173 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f499b5e4550248b6b408d516e5cea4c5" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 38889 } } peers { permanent_uuid: "7b056c9cca154ce48cc5646e1df16173" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 45365 } } peers { permanent_uuid: "15d976393f624eb4b5f44f848a656b92" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 39711 } }
I20260806 03:46:51.590626 14461 raft_consensus.cc:348] T a26a93f800674986b06b4ce2c27f4a4a P 15d976393f624eb4b5f44f848a656b92 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f499b5e4550248b6b408d516e5cea4c5" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 38889 } } peers { permanent_uuid: "7b056c9cca154ce48cc5646e1df16173" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 45365 } } peers { permanent_uuid: "15d976393f624eb4b5f44f848a656b92" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 39711 } }
I20260806 03:46:51.590700 14460 raft_consensus.cc:374] T a26a93f800674986b06b4ce2c27f4a4a P 7b056c9cca154ce48cc5646e1df16173 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:46:51.590709 14461 raft_consensus.cc:374] T a26a93f800674986b06b4ce2c27f4a4a P 15d976393f624eb4b5f44f848a656b92 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:46:51.590728 14460 raft_consensus.cc:729] T a26a93f800674986b06b4ce2c27f4a4a P 7b056c9cca154ce48cc5646e1df16173 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7b056c9cca154ce48cc5646e1df16173, State: Initialized, Role: FOLLOWER
I20260806 03:46:51.590734 14461 raft_consensus.cc:729] T a26a93f800674986b06b4ce2c27f4a4a P 15d976393f624eb4b5f44f848a656b92 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 15d976393f624eb4b5f44f848a656b92, State: Initialized, Role: FOLLOWER
I20260806 03:46:51.590783 14460 consensus_queue.cc:260] T a26a93f800674986b06b4ce2c27f4a4a P 7b056c9cca154ce48cc5646e1df16173 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f499b5e4550248b6b408d516e5cea4c5" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 38889 } } peers { permanent_uuid: "7b056c9cca154ce48cc5646e1df16173" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 45365 } } peers { permanent_uuid: "15d976393f624eb4b5f44f848a656b92" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 39711 } }
I20260806 03:46:51.590807 14461 consensus_queue.cc:260] T a26a93f800674986b06b4ce2c27f4a4a P 15d976393f624eb4b5f44f848a656b92 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f499b5e4550248b6b408d516e5cea4c5" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 38889 } } peers { permanent_uuid: "7b056c9cca154ce48cc5646e1df16173" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 45365 } } peers { permanent_uuid: "15d976393f624eb4b5f44f848a656b92" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 39711 } }
I20260806 03:46:51.590880 14460 ts_tablet_manager.cc:1428] T a26a93f800674986b06b4ce2c27f4a4a P 7b056c9cca154ce48cc5646e1df16173: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260806 03:46:51.590905 14461 ts_tablet_manager.cc:1428] T a26a93f800674986b06b4ce2c27f4a4a P 15d976393f624eb4b5f44f848a656b92: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
W20260806 03:46:51.647071 14435 tablet.cc:2369] T a26a93f800674986b06b4ce2c27f4a4a P 15d976393f624eb4b5f44f848a656b92: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 03:46:51.766078 14471 raft_consensus.cc:482] T a26a93f800674986b06b4ce2c27f4a4a P f499b5e4550248b6b408d516e5cea4c5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 03:46:51.766228 14471 raft_consensus.cc:504] T a26a93f800674986b06b4ce2c27f4a4a P f499b5e4550248b6b408d516e5cea4c5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f499b5e4550248b6b408d516e5cea4c5" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 38889 } } peers { permanent_uuid: "7b056c9cca154ce48cc5646e1df16173" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 45365 } } peers { permanent_uuid: "15d976393f624eb4b5f44f848a656b92" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 39711 } }
I20260806 03:46:51.766547 14468 raft_consensus.cc:482] T a26a93f800674986b06b4ce2c27f4a4a P 15d976393f624eb4b5f44f848a656b92 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 03:46:51.766644 14468 raft_consensus.cc:504] T a26a93f800674986b06b4ce2c27f4a4a P 15d976393f624eb4b5f44f848a656b92 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f499b5e4550248b6b408d516e5cea4c5" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 38889 } } peers { permanent_uuid: "7b056c9cca154ce48cc5646e1df16173" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 45365 } } peers { permanent_uuid: "15d976393f624eb4b5f44f848a656b92" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 39711 } }
I20260806 03:46:51.766872 14471 leader_election.cc:288] T a26a93f800674986b06b4ce2c27f4a4a P f499b5e4550248b6b408d516e5cea4c5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7b056c9cca154ce48cc5646e1df16173 (127.11.253.194:45365), 15d976393f624eb4b5f44f848a656b92 (127.11.253.195:39711)
I20260806 03:46:51.767133 14468 leader_election.cc:288] T a26a93f800674986b06b4ce2c27f4a4a P 15d976393f624eb4b5f44f848a656b92 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f499b5e4550248b6b408d516e5cea4c5 (127.11.253.193:38889), 7b056c9cca154ce48cc5646e1df16173 (127.11.253.194:45365)
I20260806 03:46:51.771287 14288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:46:51.767325 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.193:38889 (local address 127.11.253.195:48071)
0806 03:46:51.767372 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0806 03:46:51.767387 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 03:46:51.767466 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:46:51.767699 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:46:51.767700 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:46:51.767701 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 03:46:51.767794 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:46:51.767796 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:46:51.768738 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:46:51.768739 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:46:51.769989 (+ 1250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:46:51.769992 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:46:51.770982 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:46:51.770983 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:46:51.771073 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:46:51.771078 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 03:46:51.771135 (+ 57us) client_negotiation.cc:240] Negotiation successful
0806 03:46:51.771180 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":24}
I20260806 03:46:51.771443 14507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:46:51.767479 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.253.195:51887 (local address 127.11.253.194:45365)
0806 03:46:51.767578 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0806 03:46:51.767580 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:46:51.767589 (+ 9us) server_negotiation.cc:407] Connection header received
0806 03:46:51.767655 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:46:51.767656 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:46:51.767667 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:46:51.767732 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 03:46:51.767933 (+ 201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:46:51.768807 (+ 874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:46:51.770300 (+ 1493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:46:51.770907 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:46:51.770977 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:46:51.770984 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0806 03:46:51.771323 (+ 339us) server_negotiation.cc:299] Negotiation successful
0806 03:46:51.771372 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82,"thread_start_us":47,"threads_started":1}
I20260806 03:46:51.771438 14504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:46:51.767108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.195:39711 (local address 127.11.253.193:56367)
0806 03:46:51.767628 (+ 520us) negotiation.cc:107] Waiting for socket to connect
0806 03:46:51.767639 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 03:46:51.767779 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:46:51.767943 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:46:51.767944 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:46:51.767946 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 03:46:51.768016 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:46:51.768018 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:46:51.769008 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:46:51.769009 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:46:51.770350 (+ 1341us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:46:51.770353 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:46:51.771063 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:46:51.771064 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:46:51.771136 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:46:51.771141 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 03:46:51.771304 (+ 163us) client_negotiation.cc:240] Negotiation successful
0806 03:46:51.771349 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":509,"mutex_wait_us":8,"thread_start_us":81,"threads_started":1}
I20260806 03:46:51.771539 14458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:46:51.767289 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.253.193:56367 (local address 127.11.253.195:39711)
0806 03:46:51.767331 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0806 03:46:51.767333 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:46:51.767714 (+ 381us) server_negotiation.cc:407] Connection header received
0806 03:46:51.767835 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:46:51.767836 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:46:51.767843 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:46:51.767896 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 03:46:51.768093 (+ 197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:46:51.768819 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:46:51.770436 (+ 1617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:46:51.770931 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:46:51.770972 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:46:51.770980 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0806 03:46:51.771218 (+ 238us) server_negotiation.cc:299] Negotiation successful
0806 03:46:51.771265 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260806 03:46:51.771636 14456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:46:51.767166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.253.193:37307 (local address 127.11.253.194:45365)
0806 03:46:51.767287 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0806 03:46:51.767290 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 03:46:51.767590 (+ 300us) server_negotiation.cc:407] Connection header received
0806 03:46:51.767899 (+ 309us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:46:51.767901 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:46:51.767914 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:46:51.767948 (+ 34us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 03:46:51.768216 (+ 268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:46:51.768915 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:46:51.770684 (+ 1769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:46:51.771205 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:46:51.771253 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:46:51.771261 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0806 03:46:51.771515 (+ 254us) server_negotiation.cc:299] Negotiation successful
0806 03:46:51.771559 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260806 03:46:51.771787 14459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:46:51.767395 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.253.195:48071 (local address 127.11.253.193:38889)
0806 03:46:51.767437 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0806 03:46:51.767439 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:46:51.767451 (+ 12us) server_negotiation.cc:407] Connection header received
0806 03:46:51.767562 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:46:51.767563 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:46:51.767574 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:46:51.767630 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 03:46:51.767880 (+ 250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:46:51.768626 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:46:51.770101 (+ 1475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:46:51.770596 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:46:51.770665 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:46:51.770687 (+ 22us) server_negotiation.cc:1035] Waiting for connection context
0806 03:46:51.771666 (+ 979us) server_negotiation.cc:299] Negotiation successful
0806 03:46:51.771727 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 03:46:51.771744 14374 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a26a93f800674986b06b4ce2c27f4a4a" candidate_uuid: "f499b5e4550248b6b408d516e5cea4c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "15d976393f624eb4b5f44f848a656b92" is_pre_election: true
I20260806 03:46:51.771849 14374 raft_consensus.cc:2454] T a26a93f800674986b06b4ce2c27f4a4a P 15d976393f624eb4b5f44f848a656b92 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f499b5e4550248b6b408d516e5cea4c5 in term 0.
I20260806 03:46:51.771888 14506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:46:51.767417 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.194:45365 (local address 127.11.253.195:51887)
0806 03:46:51.767530 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0806 03:46:51.767540 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 03:46:51.767597 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:46:51.767752 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:46:51.767753 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:46:51.767754 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 03:46:51.767851 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:46:51.767853 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:46:51.768905 (+ 1052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:46:51.768906 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:46:51.770197 (+ 1291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:46:51.770200 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:46:51.771043 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:46:51.771044 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:46:51.771110 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:46:51.771113 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 03:46:51.771165 (+ 52us) client_negotiation.cc:240] Negotiation successful
0806 03:46:51.771213 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102,"mutex_wait_us":3,"thread_start_us":52,"threads_started":1}
I20260806 03:46:51.772022 13997 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a26a93f800674986b06b4ce2c27f4a4a" candidate_uuid: "15d976393f624eb4b5f44f848a656b92" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f499b5e4550248b6b408d516e5cea4c5" is_pre_election: true
I20260806 03:46:51.772055 14505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:46:51.767096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.194:45365 (local address 127.11.253.193:37307)
0806 03:46:51.767523 (+ 427us) negotiation.cc:107] Waiting for socket to connect
0806 03:46:51.767533 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 03:46:51.767718 (+ 185us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:46:51.768031 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:46:51.768032 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:46:51.768034 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 03:46:51.768111 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:46:51.768113 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:46:51.769312 (+ 1199us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:46:51.769313 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:46:51.770565 (+ 1252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:46:51.770568 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:46:51.771331 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:46:51.771332 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:46:51.771416 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:46:51.771421 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 03:46:51.771742 (+ 321us) client_negotiation.cc:240] Negotiation successful
0806 03:46:51.771779 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":416,"mutex_wait_us":19,"thread_start_us":86,"threads_started":1}
I20260806 03:46:51.772107 13997 raft_consensus.cc:2454] T a26a93f800674986b06b4ce2c27f4a4a P f499b5e4550248b6b408d516e5cea4c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 15d976393f624eb4b5f44f848a656b92 in term 0.
I20260806 03:46:51.772166 14183 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a26a93f800674986b06b4ce2c27f4a4a" candidate_uuid: "15d976393f624eb4b5f44f848a656b92" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7b056c9cca154ce48cc5646e1df16173" is_pre_election: true
I20260806 03:46:51.772339 14183 raft_consensus.cc:2454] T a26a93f800674986b06b4ce2c27f4a4a P 7b056c9cca154ce48cc5646e1df16173 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 15d976393f624eb4b5f44f848a656b92 in term 0.
I20260806 03:46:51.772392 14276 leader_election.cc:302] T a26a93f800674986b06b4ce2c27f4a4a P 15d976393f624eb4b5f44f848a656b92 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 15d976393f624eb4b5f44f848a656b92, f499b5e4550248b6b408d516e5cea4c5; no voters:
I20260806 03:46:51.772400 14182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a26a93f800674986b06b4ce2c27f4a4a" candidate_uuid: "f499b5e4550248b6b408d516e5cea4c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7b056c9cca154ce48cc5646e1df16173" is_pre_election: true
I20260806 03:46:51.772480 14182 raft_consensus.cc:2454] T a26a93f800674986b06b4ce2c27f4a4a P 7b056c9cca154ce48cc5646e1df16173 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f499b5e4550248b6b408d516e5cea4c5 in term 0.
I20260806 03:46:51.772545 14468 raft_consensus.cc:2781] T a26a93f800674986b06b4ce2c27f4a4a P 15d976393f624eb4b5f44f848a656b92 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 03:46:51.772591 14468 raft_consensus.cc:482] T a26a93f800674986b06b4ce2c27f4a4a P 15d976393f624eb4b5f44f848a656b92 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 03:46:51.772619 14468 raft_consensus.cc:3037] T a26a93f800674986b06b4ce2c27f4a4a P 15d976393f624eb4b5f44f848a656b92 [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:46:51.772639 13907 leader_election.cc:302] T a26a93f800674986b06b4ce2c27f4a4a P f499b5e4550248b6b408d516e5cea4c5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 15d976393f624eb4b5f44f848a656b92, f499b5e4550248b6b408d516e5cea4c5; no voters:
I20260806 03:46:51.772778 14471 raft_consensus.cc:2781] T a26a93f800674986b06b4ce2c27f4a4a P f499b5e4550248b6b408d516e5cea4c5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 03:46:51.772821 14471 raft_consensus.cc:482] T a26a93f800674986b06b4ce2c27f4a4a P f499b5e4550248b6b408d516e5cea4c5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 03:46:51.772845 14471 raft_consensus.cc:3037] T a26a93f800674986b06b4ce2c27f4a4a P f499b5e4550248b6b408d516e5cea4c5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:46:51.773469 14468 raft_consensus.cc:504] T a26a93f800674986b06b4ce2c27f4a4a P 15d976393f624eb4b5f44f848a656b92 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f499b5e4550248b6b408d516e5cea4c5" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 38889 } } peers { permanent_uuid: "7b056c9cca154ce48cc5646e1df16173" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 45365 } } peers { permanent_uuid: "15d976393f624eb4b5f44f848a656b92" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 39711 } }
I20260806 03:46:51.773500 14471 raft_consensus.cc:504] T a26a93f800674986b06b4ce2c27f4a4a P f499b5e4550248b6b408d516e5cea4c5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f499b5e4550248b6b408d516e5cea4c5" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 38889 } } peers { permanent_uuid: "7b056c9cca154ce48cc5646e1df16173" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 45365 } } peers { permanent_uuid: "15d976393f624eb4b5f44f848a656b92" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 39711 } }
I20260806 03:46:51.773623 14468 leader_election.cc:288] T a26a93f800674986b06b4ce2c27f4a4a P 15d976393f624eb4b5f44f848a656b92 [CANDIDATE]: Term 1 election: Requested vote from peers f499b5e4550248b6b408d516e5cea4c5 (127.11.253.193:38889), 7b056c9cca154ce48cc5646e1df16173 (127.11.253.194:45365)
I20260806 03:46:51.773656 14471 leader_election.cc:288] T a26a93f800674986b06b4ce2c27f4a4a P f499b5e4550248b6b408d516e5cea4c5 [CANDIDATE]: Term 1 election: Requested vote from peers 7b056c9cca154ce48cc5646e1df16173 (127.11.253.194:45365), 15d976393f624eb4b5f44f848a656b92 (127.11.253.195:39711)
I20260806 03:46:51.773808 14182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a26a93f800674986b06b4ce2c27f4a4a" candidate_uuid: "15d976393f624eb4b5f44f848a656b92" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7b056c9cca154ce48cc5646e1df16173"
I20260806 03:46:51.773818 13997 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a26a93f800674986b06b4ce2c27f4a4a" candidate_uuid: "15d976393f624eb4b5f44f848a656b92" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f499b5e4550248b6b408d516e5cea4c5"
I20260806 03:46:51.773831 14374 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a26a93f800674986b06b4ce2c27f4a4a" candidate_uuid: "f499b5e4550248b6b408d516e5cea4c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "15d976393f624eb4b5f44f848a656b92"
I20260806 03:46:51.773850 14183 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a26a93f800674986b06b4ce2c27f4a4a" candidate_uuid: "f499b5e4550248b6b408d516e5cea4c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7b056c9cca154ce48cc5646e1df16173"
I20260806 03:46:51.773902 14182 raft_consensus.cc:3037] T a26a93f800674986b06b4ce2c27f4a4a P 7b056c9cca154ce48cc5646e1df16173 [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:46:51.773923 13997 raft_consensus.cc:2379] T a26a93f800674986b06b4ce2c27f4a4a P f499b5e4550248b6b408d516e5cea4c5 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 15d976393f624eb4b5f44f848a656b92 in current term 1: Already voted for candidate f499b5e4550248b6b408d516e5cea4c5 in this term.
I20260806 03:46:51.773931 14374 raft_consensus.cc:2379] T a26a93f800674986b06b4ce2c27f4a4a P 15d976393f624eb4b5f44f848a656b92 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate f499b5e4550248b6b408d516e5cea4c5 in current term 1: Already voted for candidate 15d976393f624eb4b5f44f848a656b92 in this term.
I20260806 03:46:51.774158 13905 leader_election.cc:302] T a26a93f800674986b06b4ce2c27f4a4a P f499b5e4550248b6b408d516e5cea4c5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f499b5e4550248b6b408d516e5cea4c5; no voters: 15d976393f624eb4b5f44f848a656b92, 7b056c9cca154ce48cc5646e1df16173
I20260806 03:46:51.774273 14471 raft_consensus.cc:2726] T a26a93f800674986b06b4ce2c27f4a4a P f499b5e4550248b6b408d516e5cea4c5 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260806 03:46:51.774734 14182 raft_consensus.cc:2454] T a26a93f800674986b06b4ce2c27f4a4a P 7b056c9cca154ce48cc5646e1df16173 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 15d976393f624eb4b5f44f848a656b92 in term 1.
I20260806 03:46:51.774950 14276 leader_election.cc:302] T a26a93f800674986b06b4ce2c27f4a4a P 15d976393f624eb4b5f44f848a656b92 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 15d976393f624eb4b5f44f848a656b92, 7b056c9cca154ce48cc5646e1df16173; no voters: f499b5e4550248b6b408d516e5cea4c5
I20260806 03:46:51.775056 14468 raft_consensus.cc:2781] T a26a93f800674986b06b4ce2c27f4a4a P 15d976393f624eb4b5f44f848a656b92 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 03:46:51.775267 14468 raft_consensus.cc:686] T a26a93f800674986b06b4ce2c27f4a4a P 15d976393f624eb4b5f44f848a656b92 [term 1 LEADER]: Becoming Leader. State: Replica: 15d976393f624eb4b5f44f848a656b92, State: Running, Role: LEADER
I20260806 03:46:51.775359 14468 consensus_queue.cc:237] T a26a93f800674986b06b4ce2c27f4a4a P 15d976393f624eb4b5f44f848a656b92 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f499b5e4550248b6b408d516e5cea4c5" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 38889 } } peers { permanent_uuid: "7b056c9cca154ce48cc5646e1df16173" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 45365 } } peers { permanent_uuid: "15d976393f624eb4b5f44f848a656b92" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 39711 } }
I20260806 03:46:51.776170 13783 catalog_manager.cc:5495] T a26a93f800674986b06b4ce2c27f4a4a P 15d976393f624eb4b5f44f848a656b92 reported cstate change: term changed from 0 to 1, leader changed from <none> to 15d976393f624eb4b5f44f848a656b92 (127.11.253.195). New cstate: current_term: 1 leader_uuid: "15d976393f624eb4b5f44f848a656b92" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f499b5e4550248b6b408d516e5cea4c5" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 38889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7b056c9cca154ce48cc5646e1df16173" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 45365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "15d976393f624eb4b5f44f848a656b92" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 39711 } health_report { overall_health: HEALTHY } } }
Aug 06 03:46:51 dist-test-slave-6ntk krb5kdc[13717](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988010, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.253.254@KRBTEST.COM for kudu/127.11.253.254@KRBTEST.COM
I20260806 03:46:51.846606 14518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:46:51.838807 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:37929 (local address 127.0.0.1:52550)
0806 03:46:51.839074 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0806 03:46:51.839094 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 03:46:51.839198 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:46:51.839434 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:46:51.839435 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:46:51.839823 (+ 388us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:46:51.839873 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:46:51.839875 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:46:51.840628 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:46:51.840629 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:46:51.841215 (+ 586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:46:51.841217 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:46:51.841598 (+ 381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:46:51.841599 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:46:51.841657 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:46:51.842288 (+ 631us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:46:51.842293 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:46:51.843834 (+ 1541us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:46:51.845498 (+ 1664us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:46:51.845506 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:46:51.845508 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:46:51.845721 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:46:51.845955 (+ 234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:46:51.845956 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:46:51.845956 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:46:51.846019 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:46:51.846300 (+ 281us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:46:51.846307 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:46:51.846356 (+ 49us) client_negotiation.cc:769] Sending connection context
0806 03:46:51.846418 (+ 62us) client_negotiation.cc:240] Negotiation successful
0806 03:46:51.846522 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":233,"thread_start_us":144,"threads_started":1}
I20260806 03:46:51.846728 13852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:46:51.838909 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52550 (local address 127.11.253.254:37929)
0806 03:46:51.839038 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0806 03:46:51.839041 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 03:46:51.839175 (+ 134us) server_negotiation.cc:407] Connection header received
0806 03:46:51.839314 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:46:51.839316 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:46:51.839330 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:46:51.839410 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:46:51.839968 (+ 558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:46:51.840530 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:46:51.841310 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:46:51.841515 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:46:51.841600 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:46:51.843935 (+ 2335us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:46:51.843937 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:46:51.843937 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:46:51.843944 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:46:51.845402 (+ 1458us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:46:51.845795 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:46:51.845796 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:46:51.845796 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:46:51.845849 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:46:51.846078 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:46:51.846078 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:46:51.846078 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:46:51.846188 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:46:51.846240 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0806 03:46:51.846484 (+ 244us) server_negotiation.cc:299] Negotiation successful
0806 03:46:51.846581 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":20,"server-negotiator.queue_time_us":94}
I20260806 03:46:51.849164 13783 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.11.253.254@KRBTEST.COM'} at 127.0.0.1:52550:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260806 03:46:51.849537 13783 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 03:46:51.851313 13972 tablet_service.cc:1467] Processing CreateTablet for tablet 5846c57d9a1547d28ec05aa566d9636e (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=c5513da131ed4a11a41d3ce338d3c27e]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 03:46:51.851423 14159 tablet_service.cc:1467] Processing CreateTablet for tablet 5846c57d9a1547d28ec05aa566d9636e (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=c5513da131ed4a11a41d3ce338d3c27e]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 03:46:51.851524 13972 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5846c57d9a1547d28ec05aa566d9636e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:46:51.851511 14343 tablet_service.cc:1467] Processing CreateTablet for tablet 5846c57d9a1547d28ec05aa566d9636e (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=c5513da131ed4a11a41d3ce338d3c27e]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 03:46:51.851608 14159 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5846c57d9a1547d28ec05aa566d9636e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:46:51.851655 14343 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5846c57d9a1547d28ec05aa566d9636e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:46:51.853062 14460 tablet_bootstrap.cc:492] T 5846c57d9a1547d28ec05aa566d9636e P 7b056c9cca154ce48cc5646e1df16173: Bootstrap starting.
I20260806 03:46:51.853132 14462 tablet_bootstrap.cc:492] T 5846c57d9a1547d28ec05aa566d9636e P f499b5e4550248b6b408d516e5cea4c5: Bootstrap starting.
I20260806 03:46:51.853396 14460 tablet_bootstrap.cc:654] T 5846c57d9a1547d28ec05aa566d9636e P 7b056c9cca154ce48cc5646e1df16173: Neither blocks nor log segments found. Creating new log.
I20260806 03:46:51.853507 14462 tablet_bootstrap.cc:654] T 5846c57d9a1547d28ec05aa566d9636e P f499b5e4550248b6b408d516e5cea4c5: Neither blocks nor log segments found. Creating new log.
I20260806 03:46:51.853956 14461 tablet_bootstrap.cc:492] T 5846c57d9a1547d28ec05aa566d9636e P 15d976393f624eb4b5f44f848a656b92: Bootstrap starting.
I20260806 03:46:51.854025 14460 tablet_bootstrap.cc:492] T 5846c57d9a1547d28ec05aa566d9636e P 7b056c9cca154ce48cc5646e1df16173: No bootstrap required, opened a new log
I20260806 03:46:51.854072 14460 ts_tablet_manager.cc:1397] T 5846c57d9a1547d28ec05aa566d9636e P 7b056c9cca154ce48cc5646e1df16173: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260806 03:46:51.854249 14460 raft_consensus.cc:348] T 5846c57d9a1547d28ec05aa566d9636e P 7b056c9cca154ce48cc5646e1df16173 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f499b5e4550248b6b408d516e5cea4c5" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 38889 } } peers { permanent_uuid: "7b056c9cca154ce48cc5646e1df16173" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 45365 } } peers { permanent_uuid: "15d976393f624eb4b5f44f848a656b92" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 39711 } }
I20260806 03:46:51.854324 14460 raft_consensus.cc:374] T 5846c57d9a1547d28ec05aa566d9636e P 7b056c9cca154ce48cc5646e1df16173 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:46:51.854339 14460 raft_consensus.cc:729] T 5846c57d9a1547d28ec05aa566d9636e P 7b056c9cca154ce48cc5646e1df16173 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7b056c9cca154ce48cc5646e1df16173, State: Initialized, Role: FOLLOWER
I20260806 03:46:51.854343 14461 tablet_bootstrap.cc:654] T 5846c57d9a1547d28ec05aa566d9636e P 15d976393f624eb4b5f44f848a656b92: Neither blocks nor log segments found. Creating new log.
I20260806 03:46:51.854377 14460 consensus_queue.cc:260] T 5846c57d9a1547d28ec05aa566d9636e P 7b056c9cca154ce48cc5646e1df16173 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f499b5e4550248b6b408d516e5cea4c5" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 38889 } } peers { permanent_uuid: "7b056c9cca154ce48cc5646e1df16173" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 45365 } } peers { permanent_uuid: "15d976393f624eb4b5f44f848a656b92" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 39711 } }
I20260806 03:46:51.854497 14466 tablet_replica.cc:439] T 5846c57d9a1547d28ec05aa566d9636e P 7b056c9cca154ce48cc5646e1df16173: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f499b5e4550248b6b408d516e5cea4c5" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 38889 } } peers { permanent_uuid: "7b056c9cca154ce48cc5646e1df16173" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 45365 } } peers { permanent_uuid: "15d976393f624eb4b5f44f848a656b92" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 39711 } } }
I20260806 03:46:51.854583 14466 tablet_replica.cc:442] T 5846c57d9a1547d28ec05aa566d9636e P 7b056c9cca154ce48cc5646e1df16173: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 03:46:51.854954 14462 tablet_bootstrap.cc:492] T 5846c57d9a1547d28ec05aa566d9636e P f499b5e4550248b6b408d516e5cea4c5: No bootstrap required, opened a new log
I20260806 03:46:51.855032 14462 ts_tablet_manager.cc:1397] T 5846c57d9a1547d28ec05aa566d9636e P f499b5e4550248b6b408d516e5cea4c5: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260806 03:46:51.854507 14460 ts_tablet_manager.cc:1428] T 5846c57d9a1547d28ec05aa566d9636e P 7b056c9cca154ce48cc5646e1df16173: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 03:46:51.855201 14462 raft_consensus.cc:348] T 5846c57d9a1547d28ec05aa566d9636e P f499b5e4550248b6b408d516e5cea4c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f499b5e4550248b6b408d516e5cea4c5" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 38889 } } peers { permanent_uuid: "7b056c9cca154ce48cc5646e1df16173" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 45365 } } peers { permanent_uuid: "15d976393f624eb4b5f44f848a656b92" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 39711 } }
I20260806 03:46:51.855289 14462 raft_consensus.cc:374] T 5846c57d9a1547d28ec05aa566d9636e P f499b5e4550248b6b408d516e5cea4c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:46:51.855324 14462 raft_consensus.cc:729] T 5846c57d9a1547d28ec05aa566d9636e P f499b5e4550248b6b408d516e5cea4c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f499b5e4550248b6b408d516e5cea4c5, State: Initialized, Role: FOLLOWER
I20260806 03:46:51.855365 14462 consensus_queue.cc:260] T 5846c57d9a1547d28ec05aa566d9636e P f499b5e4550248b6b408d516e5cea4c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f499b5e4550248b6b408d516e5cea4c5" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 38889 } } peers { permanent_uuid: "7b056c9cca154ce48cc5646e1df16173" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 45365 } } peers { permanent_uuid: "15d976393f624eb4b5f44f848a656b92" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 39711 } }
I20260806 03:46:51.855461 14462 ts_tablet_manager.cc:1428] T 5846c57d9a1547d28ec05aa566d9636e P f499b5e4550248b6b408d516e5cea4c5: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 03:46:51.855533 14471 tablet_replica.cc:439] T 5846c57d9a1547d28ec05aa566d9636e P f499b5e4550248b6b408d516e5cea4c5: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f499b5e4550248b6b408d516e5cea4c5" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 38889 } } peers { permanent_uuid: "7b056c9cca154ce48cc5646e1df16173" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 45365 } } peers { permanent_uuid: "15d976393f624eb4b5f44f848a656b92" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 39711 } } }
I20260806 03:46:51.855640 14471 tablet_replica.cc:442] T 5846c57d9a1547d28ec05aa566d9636e P f499b5e4550248b6b408d516e5cea4c5: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 03:46:51.855708 14461 tablet_bootstrap.cc:492] T 5846c57d9a1547d28ec05aa566d9636e P 15d976393f624eb4b5f44f848a656b92: No bootstrap required, opened a new log
I20260806 03:46:51.855762 14461 ts_tablet_manager.cc:1397] T 5846c57d9a1547d28ec05aa566d9636e P 15d976393f624eb4b5f44f848a656b92: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 03:46:51.856041 14461 raft_consensus.cc:348] T 5846c57d9a1547d28ec05aa566d9636e P 15d976393f624eb4b5f44f848a656b92 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f499b5e4550248b6b408d516e5cea4c5" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 38889 } } peers { permanent_uuid: "7b056c9cca154ce48cc5646e1df16173" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 45365 } } peers { permanent_uuid: "15d976393f624eb4b5f44f848a656b92" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 39711 } }
I20260806 03:46:51.856123 14461 raft_consensus.cc:374] T 5846c57d9a1547d28ec05aa566d9636e P 15d976393f624eb4b5f44f848a656b92 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:46:51.856148 14461 raft_consensus.cc:729] T 5846c57d9a1547d28ec05aa566d9636e P 15d976393f624eb4b5f44f848a656b92 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 15d976393f624eb4b5f44f848a656b92, State: Initialized, Role: FOLLOWER
I20260806 03:46:51.856206 14461 consensus_queue.cc:260] T 5846c57d9a1547d28ec05aa566d9636e P 15d976393f624eb4b5f44f848a656b92 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f499b5e4550248b6b408d516e5cea4c5" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 38889 } } peers { permanent_uuid: "7b056c9cca154ce48cc5646e1df16173" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 45365 } } peers { permanent_uuid: "15d976393f624eb4b5f44f848a656b92" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 39711 } }
I20260806 03:46:51.856304 14461 ts_tablet_manager.cc:1428] T 5846c57d9a1547d28ec05aa566d9636e P 15d976393f624eb4b5f44f848a656b92: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 03:46:51.856371 14468 tablet_replica.cc:439] T 5846c57d9a1547d28ec05aa566d9636e P 15d976393f624eb4b5f44f848a656b92: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f499b5e4550248b6b408d516e5cea4c5" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 38889 } } peers { permanent_uuid: "7b056c9cca154ce48cc5646e1df16173" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 45365 } } peers { permanent_uuid: "15d976393f624eb4b5f44f848a656b92" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 39711 } } }
I20260806 03:46:51.856472 14468 tablet_replica.cc:442] T 5846c57d9a1547d28ec05aa566d9636e P 15d976393f624eb4b5f44f848a656b92: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 03:46:51.879722 14468 raft_consensus.cc:482] T 5846c57d9a1547d28ec05aa566d9636e P 15d976393f624eb4b5f44f848a656b92 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 03:46:51.879823 14468 raft_consensus.cc:504] T 5846c57d9a1547d28ec05aa566d9636e P 15d976393f624eb4b5f44f848a656b92 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f499b5e4550248b6b408d516e5cea4c5" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 38889 } } peers { permanent_uuid: "7b056c9cca154ce48cc5646e1df16173" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 45365 } } peers { permanent_uuid: "15d976393f624eb4b5f44f848a656b92" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 39711 } }
I20260806 03:46:51.880004 14468 leader_election.cc:288] T 5846c57d9a1547d28ec05aa566d9636e P 15d976393f624eb4b5f44f848a656b92 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f499b5e4550248b6b408d516e5cea4c5 (127.11.253.193:38889), 7b056c9cca154ce48cc5646e1df16173 (127.11.253.194:45365)
I20260806 03:46:51.880306 14182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5846c57d9a1547d28ec05aa566d9636e" candidate_uuid: "15d976393f624eb4b5f44f848a656b92" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7b056c9cca154ce48cc5646e1df16173" is_pre_election: true
I20260806 03:46:51.880314 13997 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5846c57d9a1547d28ec05aa566d9636e" candidate_uuid: "15d976393f624eb4b5f44f848a656b92" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f499b5e4550248b6b408d516e5cea4c5" is_pre_election: true
I20260806 03:46:51.880465 13997 raft_consensus.cc:2454] T 5846c57d9a1547d28ec05aa566d9636e P f499b5e4550248b6b408d516e5cea4c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 15d976393f624eb4b5f44f848a656b92 in term 0.
I20260806 03:46:51.880467 14182 raft_consensus.cc:2454] T 5846c57d9a1547d28ec05aa566d9636e P 7b056c9cca154ce48cc5646e1df16173 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 15d976393f624eb4b5f44f848a656b92 in term 0.
I20260806 03:46:51.880637 14276 leader_election.cc:302] T 5846c57d9a1547d28ec05aa566d9636e P 15d976393f624eb4b5f44f848a656b92 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 15d976393f624eb4b5f44f848a656b92, 7b056c9cca154ce48cc5646e1df16173; no voters:
I20260806 03:46:51.880790 14468 raft_consensus.cc:2781] T 5846c57d9a1547d28ec05aa566d9636e P 15d976393f624eb4b5f44f848a656b92 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 03:46:51.880861 14468 raft_consensus.cc:482] T 5846c57d9a1547d28ec05aa566d9636e P 15d976393f624eb4b5f44f848a656b92 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 03:46:51.880904 14468 raft_consensus.cc:3037] T 5846c57d9a1547d28ec05aa566d9636e P 15d976393f624eb4b5f44f848a656b92 [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:46:51.881649 14468 raft_consensus.cc:504] T 5846c57d9a1547d28ec05aa566d9636e P 15d976393f624eb4b5f44f848a656b92 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f499b5e4550248b6b408d516e5cea4c5" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 38889 } } peers { permanent_uuid: "7b056c9cca154ce48cc5646e1df16173" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 45365 } } peers { permanent_uuid: "15d976393f624eb4b5f44f848a656b92" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 39711 } }
I20260806 03:46:51.881799 14468 leader_election.cc:288] T 5846c57d9a1547d28ec05aa566d9636e P 15d976393f624eb4b5f44f848a656b92 [CANDIDATE]: Term 1 election: Requested vote from peers f499b5e4550248b6b408d516e5cea4c5 (127.11.253.193:38889), 7b056c9cca154ce48cc5646e1df16173 (127.11.253.194:45365)
I20260806 03:46:51.882052 13997 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5846c57d9a1547d28ec05aa566d9636e" candidate_uuid: "15d976393f624eb4b5f44f848a656b92" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f499b5e4550248b6b408d516e5cea4c5"
I20260806 03:46:51.882148 13997 raft_consensus.cc:3037] T 5846c57d9a1547d28ec05aa566d9636e P f499b5e4550248b6b408d516e5cea4c5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:46:51.882287 14182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5846c57d9a1547d28ec05aa566d9636e" candidate_uuid: "15d976393f624eb4b5f44f848a656b92" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7b056c9cca154ce48cc5646e1df16173"
I20260806 03:46:51.882405 14182 raft_consensus.cc:3037] T 5846c57d9a1547d28ec05aa566d9636e P 7b056c9cca154ce48cc5646e1df16173 [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:46:51.882880 13997 raft_consensus.cc:2454] T 5846c57d9a1547d28ec05aa566d9636e P f499b5e4550248b6b408d516e5cea4c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 15d976393f624eb4b5f44f848a656b92 in term 1.
I20260806 03:46:51.883054 14276 leader_election.cc:302] T 5846c57d9a1547d28ec05aa566d9636e P 15d976393f624eb4b5f44f848a656b92 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 15d976393f624eb4b5f44f848a656b92, f499b5e4550248b6b408d516e5cea4c5; no voters:
I20260806 03:46:51.883164 14468 raft_consensus.cc:2781] T 5846c57d9a1547d28ec05aa566d9636e P 15d976393f624eb4b5f44f848a656b92 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 03:46:51.883172 14182 raft_consensus.cc:2454] T 5846c57d9a1547d28ec05aa566d9636e P 7b056c9cca154ce48cc5646e1df16173 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 15d976393f624eb4b5f44f848a656b92 in term 1.
I20260806 03:46:51.883241 14468 raft_consensus.cc:686] T 5846c57d9a1547d28ec05aa566d9636e P 15d976393f624eb4b5f44f848a656b92 [term 1 LEADER]: Becoming Leader. State: Replica: 15d976393f624eb4b5f44f848a656b92, State: Running, Role: LEADER
I20260806 03:46:51.883299 14468 consensus_queue.cc:237] T 5846c57d9a1547d28ec05aa566d9636e P 15d976393f624eb4b5f44f848a656b92 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f499b5e4550248b6b408d516e5cea4c5" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 38889 } } peers { permanent_uuid: "7b056c9cca154ce48cc5646e1df16173" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 45365 } } peers { permanent_uuid: "15d976393f624eb4b5f44f848a656b92" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 39711 } }
I20260806 03:46:51.883667 14508 tablet_replica.cc:439] T 5846c57d9a1547d28ec05aa566d9636e P 15d976393f624eb4b5f44f848a656b92: TxnStatusTablet state changed. Reason: New leader 15d976393f624eb4b5f44f848a656b92. Latest consensus state: current_term: 1 leader_uuid: "15d976393f624eb4b5f44f848a656b92" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f499b5e4550248b6b408d516e5cea4c5" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 38889 } } peers { permanent_uuid: "7b056c9cca154ce48cc5646e1df16173" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 45365 } } peers { permanent_uuid: "15d976393f624eb4b5f44f848a656b92" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 39711 } } }
I20260806 03:46:51.883754 14508 tablet_replica.cc:442] T 5846c57d9a1547d28ec05aa566d9636e P 15d976393f624eb4b5f44f848a656b92: This TxnStatusTablet replica's current role is: LEADER
I20260806 03:46:51.883989 13783 catalog_manager.cc:5495] T 5846c57d9a1547d28ec05aa566d9636e P 15d976393f624eb4b5f44f848a656b92 reported cstate change: term changed from 0 to 1, leader changed from <none> to 15d976393f624eb4b5f44f848a656b92 (127.11.253.195). New cstate: current_term: 1 leader_uuid: "15d976393f624eb4b5f44f848a656b92" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f499b5e4550248b6b408d516e5cea4c5" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 38889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7b056c9cca154ce48cc5646e1df16173" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 45365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "15d976393f624eb4b5f44f848a656b92" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 39711 } health_report { overall_health: HEALTHY } } }
I20260806 03:46:51.884327 14524 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260806 03:46:51.884476 14524 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260806 03:46:51.893565 14458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:46:51.890859 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41002 (local address 127.11.253.195:39711)
0806 03:46:51.890929 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0806 03:46:51.890931 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:46:51.890940 (+ 9us) server_negotiation.cc:407] Connection header received
0806 03:46:51.890978 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:46:51.890979 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:46:51.891000 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:46:51.891081 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 03:46:51.891184 (+ 103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:46:51.891928 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:46:51.892652 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:46:51.892961 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:46:51.893036 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:46:51.893198 (+ 162us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 03:46:51.893323 (+ 125us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 03:46:51.893403 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0806 03:46:51.893444 (+ 41us) server_negotiation.cc:299] Negotiation successful
0806 03:46:51.893467 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260806 03:46:51.893537 14518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:46:51.890733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.195:39711 (local address 127.0.0.1:41002)
0806 03:46:51.890814 (+ 81us) negotiation.cc:107] Waiting for socket to connect
0806 03:46:51.890828 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 03:46:51.890897 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:46:51.891090 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:46:51.891091 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:46:51.891093 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 03:46:51.891138 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:46:51.891140 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:46:51.892006 (+ 866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:46:51.892007 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:46:51.892569 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:46:51.892571 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:46:51.893042 (+ 471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:46:51.893042 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:46:51.893089 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:46:51.893104 (+ 15us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 03:46:51.893394 (+ 290us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 03:46:51.893401 (+ 7us) client_negotiation.cc:769] Sending connection context
0806 03:46:51.893437 (+ 36us) client_negotiation.cc:240] Negotiation successful
0806 03:46:51.893479 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40}
I20260806 03:46:51.894879 13997 raft_consensus.cc:1261] T 5846c57d9a1547d28ec05aa566d9636e P f499b5e4550248b6b408d516e5cea4c5 [term 1 FOLLOWER]: Refusing update from remote peer 15d976393f624eb4b5f44f848a656b92: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 03:46:51.894901 14182 raft_consensus.cc:1261] T 5846c57d9a1547d28ec05aa566d9636e P 7b056c9cca154ce48cc5646e1df16173 [term 1 FOLLOWER]: Refusing update from remote peer 15d976393f624eb4b5f44f848a656b92: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 03:46:51.895126 14508 consensus_queue.cc:1035] T 5846c57d9a1547d28ec05aa566d9636e P 15d976393f624eb4b5f44f848a656b92 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f499b5e4550248b6b408d516e5cea4c5" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 38889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 03:46:51.895274 14468 consensus_queue.cc:1035] T 5846c57d9a1547d28ec05aa566d9636e P 15d976393f624eb4b5f44f848a656b92 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7b056c9cca154ce48cc5646e1df16173" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 45365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 03:46:51.895974 14466 tablet_replica.cc:439] T 5846c57d9a1547d28ec05aa566d9636e P 7b056c9cca154ce48cc5646e1df16173: TxnStatusTablet state changed. Reason: New leader 15d976393f624eb4b5f44f848a656b92. Latest consensus state: current_term: 1 leader_uuid: "15d976393f624eb4b5f44f848a656b92" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f499b5e4550248b6b408d516e5cea4c5" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 38889 } } peers { permanent_uuid: "7b056c9cca154ce48cc5646e1df16173" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 45365 } } peers { permanent_uuid: "15d976393f624eb4b5f44f848a656b92" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 39711 } } }
I20260806 03:46:51.896076 14466 tablet_replica.cc:442] T 5846c57d9a1547d28ec05aa566d9636e P 7b056c9cca154ce48cc5646e1df16173: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 03:46:51.896210 14471 tablet_replica.cc:439] T 5846c57d9a1547d28ec05aa566d9636e P f499b5e4550248b6b408d516e5cea4c5: TxnStatusTablet state changed. Reason: New leader 15d976393f624eb4b5f44f848a656b92. Latest consensus state: current_term: 1 leader_uuid: "15d976393f624eb4b5f44f848a656b92" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f499b5e4550248b6b408d516e5cea4c5" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 38889 } } peers { permanent_uuid: "7b056c9cca154ce48cc5646e1df16173" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 45365 } } peers { permanent_uuid: "15d976393f624eb4b5f44f848a656b92" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 39711 } } }
I20260806 03:46:51.896311 14471 tablet_replica.cc:442] T 5846c57d9a1547d28ec05aa566d9636e P f499b5e4550248b6b408d516e5cea4c5: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 03:46:51.897032 14489 mvcc.cc:205] Tried to move back new op lower bound from 7315406896719380480 to 7315406896674521088. Current Snapshot: MvccSnapshot[applied={T|T < 7315406896719380480}]
I20260806 03:46:51.897450 14482 mvcc.cc:205] Tried to move back new op lower bound from 7315406896719380480 to 7315406896674521088. Current Snapshot: MvccSnapshot[applied={T|T < 7315406896719380480}]
I20260806 03:46:51.897558 14471 tablet_replica.cc:439] T 5846c57d9a1547d28ec05aa566d9636e P f499b5e4550248b6b408d516e5cea4c5: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "15d976393f624eb4b5f44f848a656b92" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f499b5e4550248b6b408d516e5cea4c5" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 38889 } } peers { permanent_uuid: "7b056c9cca154ce48cc5646e1df16173" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 45365 } } peers { permanent_uuid: "15d976393f624eb4b5f44f848a656b92" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 39711 } } }
I20260806 03:46:51.897651 14471 tablet_replica.cc:442] T 5846c57d9a1547d28ec05aa566d9636e P f499b5e4550248b6b408d516e5cea4c5: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 03:46:51.897785 14468 tablet_replica.cc:439] T 5846c57d9a1547d28ec05aa566d9636e P 15d976393f624eb4b5f44f848a656b92: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "15d976393f624eb4b5f44f848a656b92" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f499b5e4550248b6b408d516e5cea4c5" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 38889 } } peers { permanent_uuid: "7b056c9cca154ce48cc5646e1df16173" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 45365 } } peers { permanent_uuid: "15d976393f624eb4b5f44f848a656b92" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 39711 } } }
I20260806 03:46:51.897887 14468 tablet_replica.cc:442] T 5846c57d9a1547d28ec05aa566d9636e P 15d976393f624eb4b5f44f848a656b92: This TxnStatusTablet replica's current role is: LEADER
I20260806 03:46:51.898047 14531 tablet_replica.cc:439] T 5846c57d9a1547d28ec05aa566d9636e P 15d976393f624eb4b5f44f848a656b92: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "15d976393f624eb4b5f44f848a656b92" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f499b5e4550248b6b408d516e5cea4c5" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 38889 } } peers { permanent_uuid: "7b056c9cca154ce48cc5646e1df16173" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 45365 } } peers { permanent_uuid: "15d976393f624eb4b5f44f848a656b92" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 39711 } } }
I20260806 03:46:51.898361 14466 tablet_replica.cc:439] T 5846c57d9a1547d28ec05aa566d9636e P 7b056c9cca154ce48cc5646e1df16173: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "15d976393f624eb4b5f44f848a656b92" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f499b5e4550248b6b408d516e5cea4c5" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 38889 } } peers { permanent_uuid: "7b056c9cca154ce48cc5646e1df16173" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 45365 } } peers { permanent_uuid: "15d976393f624eb4b5f44f848a656b92" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 39711 } } }
I20260806 03:46:51.898489 14466 tablet_replica.cc:442] T 5846c57d9a1547d28ec05aa566d9636e P 7b056c9cca154ce48cc5646e1df16173: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 03:46:51.898646 14531 tablet_replica.cc:442] T 5846c57d9a1547d28ec05aa566d9636e P 15d976393f624eb4b5f44f848a656b92: This TxnStatusTablet replica's current role is: LEADER
I20260806 03:46:51.901834 14458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:46:51.899540 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41004 (local address 127.11.253.195:39711)
0806 03:46:51.899592 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0806 03:46:51.899594 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:46:51.899619 (+ 25us) server_negotiation.cc:407] Connection header received
0806 03:46:51.899684 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:46:51.899685 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:46:51.899695 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:46:51.899748 (+ 53us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 03:46:51.899869 (+ 121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:46:51.900429 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:46:51.901143 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:46:51.901349 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:46:51.901429 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:46:51.901587 (+ 158us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 03:46:51.901620 (+ 33us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 03:46:51.901670 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0806 03:46:51.901740 (+ 70us) server_negotiation.cc:299] Negotiation successful
0806 03:46:51.901759 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 03:46:51.906914 14458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:46:51.904512 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.253.195:34363 (local address 127.11.253.195:39711)
0806 03:46:51.904563 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0806 03:46:51.904564 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 03:46:51.904571 (+ 7us) server_negotiation.cc:407] Connection header received
0806 03:46:51.904588 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:46:51.904589 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:46:51.904597 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:46:51.904668 (+ 71us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 03:46:51.904813 (+ 145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:46:51.905318 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:46:51.906180 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:46:51.906384 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:46:51.906469 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:46:51.906650 (+ 181us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 03:46:51.906681 (+ 31us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 03:46:51.906729 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0806 03:46:51.906784 (+ 55us) server_negotiation.cc:299] Negotiation successful
0806 03:46:51.906808 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":10,"server-negotiator.queue_time_us":34}
I20260806 03:46:51.906914 14443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:46:51.904317 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.195:39711 (local address 127.11.253.195:34363)
0806 03:46:51.904412 (+ 95us) negotiation.cc:107] Waiting for socket to connect
0806 03:46:51.904421 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 03:46:51.904452 (+ 31us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:46:51.904669 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:46:51.904670 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:46:51.904671 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 03:46:51.904748 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:46:51.904749 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:46:51.905395 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:46:51.905396 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:46:51.906086 (+ 690us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:46:51.906088 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:46:51.906471 (+ 383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:46:51.906472 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:46:51.906541 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:46:51.906554 (+ 13us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 03:46:51.906728 (+ 174us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 03:46:51.906729 (+ 1us) client_negotiation.cc:769] Sending connection context
0806 03:46:51.906780 (+ 51us) client_negotiation.cc:240] Negotiation successful
0806 03:46:51.906831 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40,"mutex_wait_us":8}
I20260806 03:46:51.909317 14182 raft_consensus.cc:1261] T a26a93f800674986b06b4ce2c27f4a4a P 7b056c9cca154ce48cc5646e1df16173 [term 1 FOLLOWER]: Refusing update from remote peer 15d976393f624eb4b5f44f848a656b92: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 03:46:51.909497 13997 raft_consensus.cc:1261] T a26a93f800674986b06b4ce2c27f4a4a P f499b5e4550248b6b408d516e5cea4c5 [term 1 FOLLOWER]: Refusing update from remote peer 15d976393f624eb4b5f44f848a656b92: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 03:46:51.909569 14531 consensus_queue.cc:1035] T a26a93f800674986b06b4ce2c27f4a4a P 15d976393f624eb4b5f44f848a656b92 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7b056c9cca154ce48cc5646e1df16173" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 45365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 03:46:51.909674 14468 consensus_queue.cc:1035] T a26a93f800674986b06b4ce2c27f4a4a P 15d976393f624eb4b5f44f848a656b92 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f499b5e4550248b6b408d516e5cea4c5" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 38889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 03:46:51.925804 13781 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52534:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 03:46:51.925964 13781 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52534:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 03:46:51.926641 13781 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2827edda0baf420a983b2c4a3c88ddaf: Sending DeleteTablet for 3 replicas of tablet a26a93f800674986b06b4ce2c27f4a4a
I20260806 03:46:51.926954 14159 tablet_service.cc:1514] Processing DeleteTablet for tablet a26a93f800674986b06b4ce2c27f4a4a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 03:46:51 UTC) from {username='kudu', principal='kudu/127.11.253.254@KRBTEST.COM'} at 127.0.0.1:59570
I20260806 03:46:51.926996 13972 tablet_service.cc:1514] Processing DeleteTablet for tablet a26a93f800674986b06b4ce2c27f4a4a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 03:46:51 UTC) from {username='kudu', principal='kudu/127.11.253.254@KRBTEST.COM'} at 127.0.0.1:53754
I20260806 03:46:51.927093 14497 tablet_replica.cc:330] T a26a93f800674986b06b4ce2c27f4a4a P f499b5e4550248b6b408d516e5cea4c5: stopping tablet replica
I20260806 03:46:51.927095 14496 tablet_replica.cc:330] T a26a93f800674986b06b4ce2c27f4a4a P 7b056c9cca154ce48cc5646e1df16173: stopping tablet replica
I20260806 03:46:51.927145 14497 raft_consensus.cc:2229] T a26a93f800674986b06b4ce2c27f4a4a P f499b5e4550248b6b408d516e5cea4c5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 03:46:51.927160 14496 raft_consensus.cc:2229] T a26a93f800674986b06b4ce2c27f4a4a P 7b056c9cca154ce48cc5646e1df16173 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 03:46:51.927186 14497 raft_consensus.cc:2258] T a26a93f800674986b06b4ce2c27f4a4a P f499b5e4550248b6b408d516e5cea4c5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 03:46:51.927197 14496 raft_consensus.cc:2258] T a26a93f800674986b06b4ce2c27f4a4a P 7b056c9cca154ce48cc5646e1df16173 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 03:46:51.927564 14497 ts_tablet_manager.cc:1905] T a26a93f800674986b06b4ce2c27f4a4a P f499b5e4550248b6b408d516e5cea4c5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 03:46:51.927563 14496 ts_tablet_manager.cc:1905] T a26a93f800674986b06b4ce2c27f4a4a P 7b056c9cca154ce48cc5646e1df16173: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 03:46:51.927587 14343 tablet_service.cc:1514] Processing DeleteTablet for tablet a26a93f800674986b06b4ce2c27f4a4a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 03:46:51 UTC) from {username='kudu', principal='kudu/127.11.253.254@KRBTEST.COM'} at 127.0.0.1:40998
I20260806 03:46:51.927686 14495 tablet_replica.cc:330] T a26a93f800674986b06b4ce2c27f4a4a P 15d976393f624eb4b5f44f848a656b92: stopping tablet replica
I20260806 03:46:51.927747 14495 raft_consensus.cc:2229] T a26a93f800674986b06b4ce2c27f4a4a P 15d976393f624eb4b5f44f848a656b92 [term 1 LEADER]: Raft consensus shutting down.
I20260806 03:46:51.927840 14495 raft_consensus.cc:2258] T a26a93f800674986b06b4ce2c27f4a4a P 15d976393f624eb4b5f44f848a656b92 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 03:46:51.929319 14497 ts_tablet_manager.cc:1918] T a26a93f800674986b06b4ce2c27f4a4a P f499b5e4550248b6b408d516e5cea4c5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 03:46:51.929319 14496 ts_tablet_manager.cc:1918] T a26a93f800674986b06b4ce2c27f4a4a P 7b056c9cca154ce48cc5646e1df16173: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 03:46:51.929381 14496 log.cc:1196] T a26a93f800674986b06b4ce2c27f4a4a P 7b056c9cca154ce48cc5646e1df16173: Deleting WAL directory at /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785988004637995-12279-0/minicluster-data/ts-1/wal/wals/a26a93f800674986b06b4ce2c27f4a4a
I20260806 03:46:51.929381 14497 log.cc:1196] T a26a93f800674986b06b4ce2c27f4a4a P f499b5e4550248b6b408d516e5cea4c5: Deleting WAL directory at /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785988004637995-12279-0/minicluster-data/ts-0/wal/wals/a26a93f800674986b06b4ce2c27f4a4a
I20260806 03:46:51.929458 14495 ts_tablet_manager.cc:1905] T a26a93f800674986b06b4ce2c27f4a4a P 15d976393f624eb4b5f44f848a656b92: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 03:46:51.929615 14496 ts_tablet_manager.cc:1939] T a26a93f800674986b06b4ce2c27f4a4a P 7b056c9cca154ce48cc5646e1df16173: Deleting consensus metadata
I20260806 03:46:51.929615 14497 ts_tablet_manager.cc:1939] T a26a93f800674986b06b4ce2c27f4a4a P f499b5e4550248b6b408d516e5cea4c5: Deleting consensus metadata
I20260806 03:46:51.929986 13760 catalog_manager.cc:4841] TS f499b5e4550248b6b408d516e5cea4c5 (127.11.253.193:38889): tablet a26a93f800674986b06b4ce2c27f4a4a (table test-table [id=6c4f8eaf7746417281761c6086db798d]) successfully deleted
I20260806 03:46:51.930074 13760 catalog_manager.cc:4841] TS 7b056c9cca154ce48cc5646e1df16173 (127.11.253.194:45365): tablet a26a93f800674986b06b4ce2c27f4a4a (table test-table [id=6c4f8eaf7746417281761c6086db798d]) successfully deleted
I20260806 03:46:51.931180 14495 ts_tablet_manager.cc:1918] T a26a93f800674986b06b4ce2c27f4a4a P 15d976393f624eb4b5f44f848a656b92: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 03:46:51.931249 14495 log.cc:1196] T a26a93f800674986b06b4ce2c27f4a4a P 15d976393f624eb4b5f44f848a656b92: Deleting WAL directory at /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785988004637995-12279-0/minicluster-data/ts-2/wal/wals/a26a93f800674986b06b4ce2c27f4a4a
I20260806 03:46:51.931469 14495 ts_tablet_manager.cc:1939] T a26a93f800674986b06b4ce2c27f4a4a P 15d976393f624eb4b5f44f848a656b92: Deleting consensus metadata
I20260806 03:46:51.931842 13761 catalog_manager.cc:4841] TS 15d976393f624eb4b5f44f848a656b92 (127.11.253.195:39711): tablet a26a93f800674986b06b4ce2c27f4a4a (table test-table [id=6c4f8eaf7746417281761c6086db798d]) successfully deleted
Aug 06 03:46:51 dist-test-slave-6ntk krb5kdc[13717](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988011, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.11.253.193@KRBTEST.COM
I20260806 03:46:51.939023 14459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:46:51.928107 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53766 (local address 127.11.253.193:38889)
0806 03:46:51.928156 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0806 03:46:51.928158 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:46:51.928327 (+ 169us) server_negotiation.cc:407] Connection header received
0806 03:46:51.928395 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:46:51.928396 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:46:51.928410 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:46:51.928467 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:46:51.929040 (+ 573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:46:51.929734 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:46:51.931242 (+ 1508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:46:51.931583 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:46:51.931633 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:46:51.935116 (+ 3483us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:46:51.935129 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:46:51.935129 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:46:51.935138 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:46:51.937581 (+ 2443us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:46:51.938111 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:46:51.938115 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:46:51.938115 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:46:51.938190 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:46:51.938417 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:46:51.938417 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:46:51.938418 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:46:51.938589 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:46:51.938760 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0806 03:46:51.938809 (+ 49us) server_negotiation.cc:299] Negotiation successful
0806 03:46:51.938904 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
W20260806 03:46:51.939384 14052 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53766
I20260806 03:46:51.947902 13852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:46:51.940441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52562 (local address 127.11.253.254:37929)
0806 03:46:51.940491 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0806 03:46:51.940494 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 03:46:51.940668 (+ 174us) server_negotiation.cc:407] Connection header received
0806 03:46:51.940705 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:46:51.940706 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:46:51.940727 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:46:51.940818 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:46:51.941330 (+ 512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:46:51.942153 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:46:51.943051 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:46:51.943366 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:46:51.943435 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:46:51.944821 (+ 1386us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:46:51.944822 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:46:51.944823 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:46:51.944831 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:46:51.946723 (+ 1892us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:46:51.947056 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:46:51.947058 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:46:51.947058 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:46:51.947124 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:46:51.947308 (+ 184us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:46:51.947309 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:46:51.947310 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:46:51.947449 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:46:51.947552 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0806 03:46:51.947705 (+ 153us) server_negotiation.cc:299] Negotiation successful
0806 03:46:51.947797 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
W20260806 03:46:51.948199 13781 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52562
I20260806 03:46:51.948899 12279 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskToVWRe/build/release/bin/kudu with pid 13881
I20260806 03:46:51.962196 12279 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskToVWRe/build/release/bin/kudu with pid 14071
I20260806 03:46:51.975514 12279 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskToVWRe/build/release/bin/kudu with pid 14254
I20260806 03:46:51.988180 12279 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskToVWRe/build/release/bin/kudu with pid 13739
2026-08-06T03:46:51Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (2720 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785988004637995-12279-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 03:46:52 dist-test-slave-6ntk krb5kdc[14550](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 03:46:52 dist-test-slave-6ntk krb5kdc[14550](info): set up 2 sockets
Aug 06 03:46:52 dist-test-slave-6ntk krb5kdc[14550](info): commencing operation
krb5kdc: starting...
W20260806 03:46:53.253238 12279 mini_kdc.cc:121] Time spent starting KDC: real 1.238s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785988004637995-12279-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785988004637995-12279-0/krb5kdc/test-user.keytab.
Aug 06 03:46:53 dist-test-slave-6ntk krb5kdc[14550](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988013, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T03:46:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T03:46:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.253.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.253.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.253.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.254.keytab.
Entry for principal kudu/127.11.253.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.254.keytab.
I20260806 03:46:53.389436 12279 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskToVWRe/build/release/bin/kudu
/tmp/dist-test-taskToVWRe/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785988004637995-12279-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785988004637995-12279-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785988004637995-12279-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785988004637995-12279-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.253.254:42173
--webserver_interface=127.11.253.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.254.keytab
--principal=kudu/127.11.253.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.253.212:35741
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.253.254:42173
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785988004637995-12279-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785988004637995-12279-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785988004637995-12279-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:46:53.472994 14566 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:46:53.473235 14566 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:46:53.473272 14566 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:46:53.474481 14566 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 03:46:53.474547 14566 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260806 03:46:53.474570 14566 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:46:53.474628 14566 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:46:53.474655 14566 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 03:46:53.474684 14566 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 03:46:53.476189 14566 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.253.212:35741
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785988004637995-12279-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785988004637995-12279-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.253.254:42173
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.254.keytab
--principal=kudu/127.11.253.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.253.254:42173
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785988004637995-12279-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.253.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14566
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785988004637995-12279-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:46:53.476476 14566 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:46:53.476698 14566 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:46:53.481717 14580 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:46:53.481772 14576 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:46:53.481740 14578 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:46:53.482260 14566 server_base.cc:1034] running on GCE node
I20260806 03:46:53.482511 14566 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:46:53.482762 14566 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:46:53.483927 14566 hybrid_clock.cc:648] HybridClock initialized: now 1785988013483915 us; error 37 us; skew 500 ppm
Aug 06 03:46:53 dist-test-slave-6ntk krb5kdc[14550](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988013, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.253.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:46:53.486714 14566 init.cc:377] Logged in from keytab as kudu/127.11.253.254@KRBTEST.COM (short username kudu)
I20260806 03:46:53.487133 14566 webserver.cc:460] Webserver started at http://127.11.253.254:34575/ using document root <none> and password file <none>
I20260806 03:46:53.487349 14566 fs_manager.cc:359] Metadata directory not provided
I20260806 03:46:53.487391 14566 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:46:53.487497 14566 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:46:53.488531 14566 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785988004637995-12279-0/minicluster-data/master-0/data/instance:
uuid: "ae915b1dc3a24bdb8afeed530e88edf7"
format_stamp: "Formatted at 2026-08-06 03:46:53 on dist-test-slave-6ntk"
server_key: "5f1b9b443c7c4a3c354fe1e50d3ece58"
server_key_iv: "8acc5f86905579d9a46cfab8365150af"
server_key_version: "encryptionkey@0"
I20260806 03:46:53.488854 14566 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785988004637995-12279-0/minicluster-data/master-0/wal/instance:
uuid: "ae915b1dc3a24bdb8afeed530e88edf7"
format_stamp: "Formatted at 2026-08-06 03:46:53 on dist-test-slave-6ntk"
server_key: "5f1b9b443c7c4a3c354fe1e50d3ece58"
server_key_iv: "8acc5f86905579d9a46cfab8365150af"
server_key_version: "encryptionkey@0"
I20260806 03:46:53.490221 14566 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 03:46:53.491014 14587 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:46:53.491223 14566 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 03:46:53.491319 14566 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785988004637995-12279-0/minicluster-data/master-0/data,/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785988004637995-12279-0/minicluster-data/master-0/wal
uuid: "ae915b1dc3a24bdb8afeed530e88edf7"
format_stamp: "Formatted at 2026-08-06 03:46:53 on dist-test-slave-6ntk"
server_key: "5f1b9b443c7c4a3c354fe1e50d3ece58"
server_key_iv: "8acc5f86905579d9a46cfab8365150af"
server_key_version: "encryptionkey@0"
I20260806 03:46:53.491410 14566 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785988004637995-12279-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785988004637995-12279-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785988004637995-12279-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:46:53.508410 14566 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:46:53.510834 14566 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:46:53.510952 14566 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:46:53.515772 14566 rpc_server.cc:307] RPC server started. Bound to: 127.11.253.254:42173
I20260806 03:46:53.515822 14664 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.253.254:42173 every 8 connection(s)
I20260806 03:46:53.516168 14566 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785988004637995-12279-0/minicluster-data/master-0/data/info.pb
I20260806 03:46:53.516887 14665 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:46:53.519377 14665 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ae915b1dc3a24bdb8afeed530e88edf7: Bootstrap starting.
I20260806 03:46:53.520002 14665 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ae915b1dc3a24bdb8afeed530e88edf7: Neither blocks nor log segments found. Creating new log.
I20260806 03:46:53.520300 14665 log.cc:824] T 00000000000000000000000000000000 P ae915b1dc3a24bdb8afeed530e88edf7: Log is configured to *not* fsync() on all Append() calls
I20260806 03:46:53.521010 14665 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ae915b1dc3a24bdb8afeed530e88edf7: No bootstrap required, opened a new log
I20260806 03:46:53.523389 14665 raft_consensus.cc:348] T 00000000000000000000000000000000 P ae915b1dc3a24bdb8afeed530e88edf7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae915b1dc3a24bdb8afeed530e88edf7" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 42173 } }
I20260806 03:46:53.523559 14665 raft_consensus.cc:374] T 00000000000000000000000000000000 P ae915b1dc3a24bdb8afeed530e88edf7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:46:53.523605 14665 raft_consensus.cc:729] T 00000000000000000000000000000000 P ae915b1dc3a24bdb8afeed530e88edf7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ae915b1dc3a24bdb8afeed530e88edf7, State: Initialized, Role: FOLLOWER
I20260806 03:46:53.523797 14665 consensus_queue.cc:260] T 00000000000000000000000000000000 P ae915b1dc3a24bdb8afeed530e88edf7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae915b1dc3a24bdb8afeed530e88edf7" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 42173 } }
I20260806 03:46:53.523901 14665 raft_consensus.cc:388] T 00000000000000000000000000000000 P ae915b1dc3a24bdb8afeed530e88edf7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 03:46:53.523950 14665 raft_consensus.cc:482] T 00000000000000000000000000000000 P ae915b1dc3a24bdb8afeed530e88edf7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 03:46:53.524004 14665 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ae915b1dc3a24bdb8afeed530e88edf7 [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:46:53.524603 14665 raft_consensus.cc:504] T 00000000000000000000000000000000 P ae915b1dc3a24bdb8afeed530e88edf7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae915b1dc3a24bdb8afeed530e88edf7" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 42173 } }
I20260806 03:46:53.524730 14665 leader_election.cc:302] T 00000000000000000000000000000000 P ae915b1dc3a24bdb8afeed530e88edf7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ae915b1dc3a24bdb8afeed530e88edf7; no voters:
I20260806 03:46:53.524950 14665 leader_election.cc:288] T 00000000000000000000000000000000 P ae915b1dc3a24bdb8afeed530e88edf7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 03:46:53.524989 14669 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ae915b1dc3a24bdb8afeed530e88edf7 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 03:46:53.525020 12279 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskToVWRe/build/release/bin/kudu as pid 14566
I20260806 03:46:53.525107 12279 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785988004637995-12279-0/minicluster-data/master-0/wal/instance
I20260806 03:46:53.525117 14669 raft_consensus.cc:686] T 00000000000000000000000000000000 P ae915b1dc3a24bdb8afeed530e88edf7 [term 1 LEADER]: Becoming Leader. State: Replica: ae915b1dc3a24bdb8afeed530e88edf7, State: Running, Role: LEADER
I20260806 03:46:53.525197 14669 consensus_queue.cc:237] T 00000000000000000000000000000000 P ae915b1dc3a24bdb8afeed530e88edf7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae915b1dc3a24bdb8afeed530e88edf7" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 42173 } }
I20260806 03:46:53.525233 14665 sys_catalog.cc:564] T 00000000000000000000000000000000 P ae915b1dc3a24bdb8afeed530e88edf7 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 03:46:53.525388 12279 external_mini_cluster.cc:1428] Setting key 7531b16e165660161f65cbcf2714e472
I20260806 03:46:53.525560 14672 sys_catalog.cc:455] T 00000000000000000000000000000000 P ae915b1dc3a24bdb8afeed530e88edf7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ae915b1dc3a24bdb8afeed530e88edf7. Latest consensus state: current_term: 1 leader_uuid: "ae915b1dc3a24bdb8afeed530e88edf7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae915b1dc3a24bdb8afeed530e88edf7" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 42173 } } }
I20260806 03:46:53.525651 14672 sys_catalog.cc:458] T 00000000000000000000000000000000 P ae915b1dc3a24bdb8afeed530e88edf7 [sys.catalog]: This master's current role is: LEADER
I20260806 03:46:53.525897 14674 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 03:46:53.526621 14674 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 03:46:53.525914 14670 sys_catalog.cc:455] T 00000000000000000000000000000000 P ae915b1dc3a24bdb8afeed530e88edf7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ae915b1dc3a24bdb8afeed530e88edf7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae915b1dc3a24bdb8afeed530e88edf7" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 42173 } } }
I20260806 03:46:53.526785 14670 sys_catalog.cc:458] T 00000000000000000000000000000000 P ae915b1dc3a24bdb8afeed530e88edf7 [sys.catalog]: This master's current role is: LEADER
I20260806 03:46:53.528545 14674 catalog_manager.cc:1347] Generated new cluster ID: 2b8feeab9a584264815d0f5f2533439e
I20260806 03:46:53.528599 14674 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 03:46:53 dist-test-slave-6ntk krb5kdc[14550](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988013, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.253.254@KRBTEST.COM
I20260806 03:46:53.536605 14680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:46:53.526106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60626 (local address 127.11.253.254:42173)
0806 03:46:53.526784 (+ 678us) server_negotiation.cc:206] Beginning negotiation
0806 03:46:53.526794 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 03:46:53.526830 (+ 36us) server_negotiation.cc:407] Connection header received
0806 03:46:53.526960 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:46:53.526967 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:46:53.527029 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:46:53.527177 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:46:53.527972 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:46:53.529121 (+ 1149us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:46:53.529940 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:46:53.530413 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:46:53.530658 (+ 245us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:46:53.533060 (+ 2402us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:46:53.533065 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:46:53.533066 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:46:53.533079 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:46:53.534827 (+ 1748us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:46:53.535259 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:46:53.535263 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:46:53.535264 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:46:53.535388 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:46:53.535618 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:46:53.535619 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:46:53.535619 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:46:53.535863 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:46:53.536094 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0806 03:46:53.536167 (+ 73us) server_negotiation.cc:299] Negotiation successful
0806 03:46:53.536324 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":623,"thread_start_us":79,"threads_started":1}
I20260806 03:46:53.544003 14674 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 03:46:53.544520 14674 catalog_manager.cc:1504] Loading token signing keys...
I20260806 03:46:53.549032 14674 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ae915b1dc3a24bdb8afeed530e88edf7: Generated new TSK 0
I20260806 03:46:53.549226 14674 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.253.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.253.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.253.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.193.keytab.
Entry for principal kudu/127.11.253.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.193.keytab.
I20260806 03:46:53.586541 12279 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskToVWRe/build/release/bin/kudu
/tmp/dist-test-taskToVWRe/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785988004637995-12279-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785988004637995-12279-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785988004637995-12279-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785988004637995-12279-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.253.193:0
--local_ip_for_outbound_sockets=127.11.253.193
--webserver_interface=127.11.253.193
--webserver_port=0
--tserver_master_addrs=127.11.253.254:42173
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.193.keytab
--principal=kudu/127.11.253.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.253.212:35741
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785988004637995-12279-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785988004637995-12279-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785988004637995-12279-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:46:53.684028 14701 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:46:53.684283 14701 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:46:53.684326 14701 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:46:53.685509 14701 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:46:53.685587 14701 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:46:53.685626 14701 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260806 03:46:53.685675 14701 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 03:46:53.685714 14701 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.253.193
I20260806 03:46:53.687290 14701 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.253.212:35741
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785988004637995-12279-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785988004637995-12279-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.193.keytab
--principal=kudu/127.11.253.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.253.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785988004637995-12279-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.253.193
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.11.253.254:42173
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14701
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.253.193
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785988004637995-12279-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:46:53.687569 14701 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:46:53.687808 14701 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:46:53.692955 14707 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:46:53.692968 14708 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:46:53.693025 14701 server_base.cc:1034] running on GCE node
W20260806 03:46:53.692971 14718 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:46:53.693351 14701 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:46:53.693610 14701 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:46:53.694769 14701 hybrid_clock.cc:648] HybridClock initialized: now 1785988013694746 us; error 39 us; skew 500 ppm
Aug 06 03:46:53 dist-test-slave-6ntk krb5kdc[14550](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988013, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.253.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:46:53.697435 14701 init.cc:377] Logged in from keytab as kudu/127.11.253.193@KRBTEST.COM (short username kudu)
I20260806 03:46:53.697909 14701 webserver.cc:460] Webserver started at http://127.11.253.193:42841/ using document root <none> and password file <none>
I20260806 03:46:53.698141 14701 fs_manager.cc:359] Metadata directory not provided
I20260806 03:46:53.698208 14701 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:46:53.698371 14701 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:46:53.699419 14701 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785988004637995-12279-0/minicluster-data/ts-0/data/instance:
uuid: "2577e879d33e41ffb5a214cdae767802"
format_stamp: "Formatted at 2026-08-06 03:46:53 on dist-test-slave-6ntk"
server_key: "5f7173e9ffa9b16bc308e2bc084da0a9"
server_key_iv: "7cbce0bd505ebb7e5d162ea4f8f766f5"
server_key_version: "encryptionkey@0"
I20260806 03:46:53.699815 14701 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785988004637995-12279-0/minicluster-data/ts-0/wal/instance:
uuid: "2577e879d33e41ffb5a214cdae767802"
format_stamp: "Formatted at 2026-08-06 03:46:53 on dist-test-slave-6ntk"
server_key: "5f7173e9ffa9b16bc308e2bc084da0a9"
server_key_iv: "7cbce0bd505ebb7e5d162ea4f8f766f5"
server_key_version: "encryptionkey@0"
I20260806 03:46:53.701315 14701 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 03:46:53.702168 14734 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:46:53.702384 14701 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 03:46:53.702508 14701 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785988004637995-12279-0/minicluster-data/ts-0/data,/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785988004637995-12279-0/minicluster-data/ts-0/wal
uuid: "2577e879d33e41ffb5a214cdae767802"
format_stamp: "Formatted at 2026-08-06 03:46:53 on dist-test-slave-6ntk"
server_key: "5f7173e9ffa9b16bc308e2bc084da0a9"
server_key_iv: "7cbce0bd505ebb7e5d162ea4f8f766f5"
server_key_version: "encryptionkey@0"
I20260806 03:46:53.702610 14701 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785988004637995-12279-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785988004637995-12279-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785988004637995-12279-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:46:53.720593 14701 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:46:53.722667 14701 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:46:53.722852 14701 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:46:53.723407 14701 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 03:46:53.723474 14701 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 03:46:53.723522 14701 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 03:46:53.723544 14701 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 03:46:53 dist-test-slave-6ntk krb5kdc[14550](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988013, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.253.193@KRBTEST.COM for kudu/127.11.253.254@KRBTEST.COM
I20260806 03:46:53.731303 14701 rpc_server.cc:307] RPC server started. Bound to: 127.11.253.193:41343
I20260806 03:46:53.731405 14893 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.253.193:41343 every 8 connection(s)
I20260806 03:46:53.731755 14701 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785988004637995-12279-0/minicluster-data/ts-0/data/info.pb
I20260806 03:46:53.733325 12279 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskToVWRe/build/release/bin/kudu as pid 14701
I20260806 03:46:53.733469 12279 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785988004637995-12279-0/minicluster-data/ts-0/wal/instance
I20260806 03:46:53.733703 12279 external_mini_cluster.cc:1428] Setting key 755b59c3d5839b41e922c89622678a83
I20260806 03:46:53.734007 14680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:46:53.723954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.253.193:43749 (local address 127.11.253.254:42173)
0806 03:46:53.724028 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0806 03:46:53.724031 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 03:46:53.724245 (+ 214us) server_negotiation.cc:407] Connection header received
0806 03:46:53.724753 (+ 508us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:46:53.724755 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:46:53.724773 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:46:53.724856 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:46:53.725615 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:46:53.726420 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:46:53.727764 (+ 1344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:46:53.728101 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:46:53.728175 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:46:53.730635 (+ 2460us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:46:53.730637 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:46:53.730637 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:46:53.730645 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:46:53.732395 (+ 1750us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:46:53.732837 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:46:53.732838 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:46:53.732839 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:46:53.732891 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:46:53.733205 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:46:53.733205 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:46:53.733205 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:46:53.733308 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:46:53.733385 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0806 03:46:53.733765 (+ 380us) server_negotiation.cc:299] Negotiation successful
0806 03:46:53.733878 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260806 03:46:53.734447 14747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:46:53.723917 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:42173 (local address 127.11.253.193:43749)
0806 03:46:53.724109 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0806 03:46:53.724150 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 03:46:53.724655 (+ 505us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:46:53.724957 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:46:53.724961 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:46:53.725310 (+ 349us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:46:53.725507 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:46:53.725512 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:46:53.726516 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:46:53.726518 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:46:53.727664 (+ 1146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:46:53.727667 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:46:53.728222 (+ 555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:46:53.728223 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:46:53.728409 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:46:53.728984 (+ 575us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:46:53.729008 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:46:53.730517 (+ 1509us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:46:53.732500 (+ 1983us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:46:53.732508 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:46:53.732510 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:46:53.732761 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:46:53.732956 (+ 195us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:46:53.732957 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:46:53.732957 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:46:53.733070 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:46:53.733400 (+ 330us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:46:53.733409 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:46:53.733605 (+ 196us) client_negotiation.cc:769] Sending connection context
0806 03:46:53.733740 (+ 135us) client_negotiation.cc:240] Negotiation successful
0806 03:46:53.734106 (+ 366us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":118,"threads_started":1}
I20260806 03:46:53.734992 14894 heartbeater.cc:347] Connected to a master server at 127.11.253.254:42173
I20260806 03:46:53.735110 14894 heartbeater.cc:464] Registering TS with master...
I20260806 03:46:53.735301 14894 heartbeater.cc:511] Master 127.11.253.254:42173 requested a full tablet report, sending...
I20260806 03:46:53.735846 14606 ts_manager.cc:194] Registered new tserver with Master: 2577e879d33e41ffb5a214cdae767802 (127.11.253.193:41343)
I20260806 03:46:53.736732 14606 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.253.193@KRBTEST.COM'} at 127.11.253.193:43749
I20260806 03:46:53.742900 14902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:46:53.736153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:42173 (local address 127.11.253.193:51529)
0806 03:46:53.736300 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0806 03:46:53.736310 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 03:46:53.736389 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:46:53.736583 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:46:53.736584 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:46:53.736916 (+ 332us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:46:53.736987 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:46:53.736989 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:46:53.737971 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:46:53.737971 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:46:53.738569 (+ 598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:46:53.738571 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:46:53.738948 (+ 377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:46:53.738949 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:46:53.739025 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:46:53.739502 (+ 477us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:46:53.739507 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:46:53.739811 (+ 304us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:46:53.741786 (+ 1975us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:46:53.741787 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:46:53.741788 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:46:53.742037 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:46:53.742283 (+ 246us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:46:53.742284 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:46:53.742284 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:46:53.742348 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:46:53.742639 (+ 291us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:46:53.742640 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:46:53.742677 (+ 37us) client_negotiation.cc:769] Sending connection context
0806 03:46:53.742726 (+ 49us) client_negotiation.cc:240] Negotiation successful
0806 03:46:53.742818 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":83,"threads_started":1}
I20260806 03:46:53.742954 14680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:46:53.736206 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.253.193:51529 (local address 127.11.253.254:42173)
0806 03:46:53.736249 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0806 03:46:53.736250 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 03:46:53.736376 (+ 126us) server_negotiation.cc:407] Connection header received
0806 03:46:53.736438 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:46:53.736439 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:46:53.736451 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:46:53.736501 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:46:53.737085 (+ 584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:46:53.737816 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:46:53.738658 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:46:53.738871 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:46:53.738950 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:46:53.739923 (+ 973us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:46:53.739924 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:46:53.739924 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:46:53.739931 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:46:53.741690 (+ 1759us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:46:53.742117 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:46:53.742118 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:46:53.742118 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:46:53.742182 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:46:53.742423 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:46:53.742424 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:46:53.742424 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:46:53.742570 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:46:53.742644 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0806 03:46:53.742763 (+ 119us) server_negotiation.cc:299] Negotiation successful
0806 03:46:53.742847 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
WARNING: no policy specified for kudu/127.11.253.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.253.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.253.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.194.keytab.
Entry for principal kudu/127.11.253.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.194.keytab.
I20260806 03:46:53.765070 12279 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskToVWRe/build/release/bin/kudu
/tmp/dist-test-taskToVWRe/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785988004637995-12279-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785988004637995-12279-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785988004637995-12279-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785988004637995-12279-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.253.194:0
--local_ip_for_outbound_sockets=127.11.253.194
--webserver_interface=127.11.253.194
--webserver_port=0
--tserver_master_addrs=127.11.253.254:42173
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.194.keytab
--principal=kudu/127.11.253.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.253.212:35741
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785988004637995-12279-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785988004637995-12279-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785988004637995-12279-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:46:53.850010 14906 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:46:53.850260 14906 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:46:53.850302 14906 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:46:53.851459 14906 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:46:53.851532 14906 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:46:53.851567 14906 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260806 03:46:53.851593 14906 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 03:46:53.851634 14906 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.253.194
I20260806 03:46:53.853114 14906 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.253.212:35741
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785988004637995-12279-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785988004637995-12279-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.194.keytab
--principal=kudu/127.11.253.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.253.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785988004637995-12279-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.253.194
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.11.253.254:42173
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14906
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.253.194
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785988004637995-12279-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:46:53.853404 14906 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:46:53.853650 14906 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:46:53.858799 14925 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:46:53.858799 14922 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:46:53.858834 14920 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:46:53.859462 14906 server_base.cc:1034] running on GCE node
I20260806 03:46:53.859652 14906 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:46:53.859881 14906 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:46:53.861030 14906 hybrid_clock.cc:648] HybridClock initialized: now 1785988013861014 us; error 30 us; skew 500 ppm
Aug 06 03:46:53 dist-test-slave-6ntk krb5kdc[14550](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988013, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.253.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:46:53.863744 14906 init.cc:377] Logged in from keytab as kudu/127.11.253.194@KRBTEST.COM (short username kudu)
I20260806 03:46:53.864200 14906 webserver.cc:460] Webserver started at http://127.11.253.194:33607/ using document root <none> and password file <none>
I20260806 03:46:53.864450 14906 fs_manager.cc:359] Metadata directory not provided
I20260806 03:46:53.864529 14906 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:46:53.864677 14906 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:46:53.865760 14906 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785988004637995-12279-0/minicluster-data/ts-1/data/instance:
uuid: "74eb6a2589da4205b5e5623842ef6faa"
format_stamp: "Formatted at 2026-08-06 03:46:53 on dist-test-slave-6ntk"
server_key: "002b3ea3fe376e6ce775d342fa81157c"
server_key_iv: "e5958911b5b20fa607d8e9a02579e1e1"
server_key_version: "encryptionkey@0"
I20260806 03:46:53.866205 14906 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785988004637995-12279-0/minicluster-data/ts-1/wal/instance:
uuid: "74eb6a2589da4205b5e5623842ef6faa"
format_stamp: "Formatted at 2026-08-06 03:46:53 on dist-test-slave-6ntk"
server_key: "002b3ea3fe376e6ce775d342fa81157c"
server_key_iv: "e5958911b5b20fa607d8e9a02579e1e1"
server_key_version: "encryptionkey@0"
I20260806 03:46:53.867544 14906 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 03:46:53.868239 14934 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:46:53.868404 14906 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 03:46:53.868513 14906 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785988004637995-12279-0/minicluster-data/ts-1/data,/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785988004637995-12279-0/minicluster-data/ts-1/wal
uuid: "74eb6a2589da4205b5e5623842ef6faa"
format_stamp: "Formatted at 2026-08-06 03:46:53 on dist-test-slave-6ntk"
server_key: "002b3ea3fe376e6ce775d342fa81157c"
server_key_iv: "e5958911b5b20fa607d8e9a02579e1e1"
server_key_version: "encryptionkey@0"
I20260806 03:46:53.868587 14906 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785988004637995-12279-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785988004637995-12279-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785988004637995-12279-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:46:53.887931 14906 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:46:53.890393 14906 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:46:53.890563 14906 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:46:53.891113 14906 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 03:46:53.891185 14906 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 03:46:53.891237 14906 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 03:46:53.891276 14906 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 03:46:53.898330 14906 rpc_server.cc:307] RPC server started. Bound to: 127.11.253.194:45549
I20260806 03:46:53.898402 15101 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.253.194:45549 every 8 connection(s)
Aug 06 03:46:53 dist-test-slave-6ntk krb5kdc[14550](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988013, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.253.194@KRBTEST.COM for kudu/127.11.253.254@KRBTEST.COM
I20260806 03:46:53.898744 14906 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785988004637995-12279-0/minicluster-data/ts-1/data/info.pb
I20260806 03:46:53.900496 12279 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskToVWRe/build/release/bin/kudu as pid 14906
I20260806 03:46:53.900596 12279 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785988004637995-12279-0/minicluster-data/ts-1/wal/instance
I20260806 03:46:53.900868 12279 external_mini_cluster.cc:1428] Setting key 2a011489d41d4446cd5ff968d0ab3f56
I20260806 03:46:53.902494 14947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:46:53.891754 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:42173 (local address 127.11.253.194:45525)
0806 03:46:53.891982 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0806 03:46:53.892018 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0806 03:46:53.892547 (+ 529us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:46:53.892908 (+ 361us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:46:53.892912 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:46:53.893294 (+ 382us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:46:53.893654 (+ 360us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:46:53.893661 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:46:53.894788 (+ 1127us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:46:53.894789 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:46:53.896197 (+ 1408us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:46:53.896201 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:46:53.896646 (+ 445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:46:53.896646 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:46:53.896824 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:46:53.897404 (+ 580us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:46:53.897414 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:46:53.899193 (+ 1779us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:46:53.900966 (+ 1773us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:46:53.900974 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:46:53.900976 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:46:53.901325 (+ 349us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:46:53.901546 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:46:53.901546 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:46:53.901547 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:46:53.901687 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:46:53.902016 (+ 329us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:46:53.902021 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:46:53.902132 (+ 111us) client_negotiation.cc:769] Sending connection context
0806 03:46:53.902219 (+ 87us) client_negotiation.cc:240] Negotiation successful
0806 03:46:53.902337 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":103,"threads_started":1}
I20260806 03:46:53.902555 14680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:46:53.891772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.253.194:45525 (local address 127.11.253.254:42173)
0806 03:46:53.891831 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0806 03:46:53.891833 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:46:53.892108 (+ 275us) server_negotiation.cc:407] Connection header received
0806 03:46:53.892714 (+ 606us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:46:53.892716 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:46:53.892732 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:46:53.892825 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:46:53.893786 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:46:53.894672 (+ 886us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:46:53.896288 (+ 1616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:46:53.896545 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:46:53.896636 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:46:53.899277 (+ 2641us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:46:53.899278 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:46:53.899278 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:46:53.899285 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:46:53.900820 (+ 1535us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:46:53.901412 (+ 592us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:46:53.901413 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:46:53.901414 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:46:53.901471 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:46:53.901760 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:46:53.901761 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:46:53.901762 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:46:53.901942 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:46:53.902021 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0806 03:46:53.902319 (+ 298us) server_negotiation.cc:299] Negotiation successful
0806 03:46:53.902411 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260806 03:46:53.902966 15102 heartbeater.cc:347] Connected to a master server at 127.11.253.254:42173
I20260806 03:46:53.903065 15102 heartbeater.cc:464] Registering TS with master...
I20260806 03:46:53.903244 15102 heartbeater.cc:511] Master 127.11.253.254:42173 requested a full tablet report, sending...
I20260806 03:46:53.904029 14605 ts_manager.cc:194] Registered new tserver with Master: 74eb6a2589da4205b5e5623842ef6faa (127.11.253.194:45549)
I20260806 03:46:53.904606 14605 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.253.194@KRBTEST.COM'} at 127.11.253.194:45525
I20260806 03:46:53.910334 14680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:46:53.904185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.253.194:47755 (local address 127.11.253.254:42173)
0806 03:46:53.904227 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0806 03:46:53.904228 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 03:46:53.904261 (+ 33us) server_negotiation.cc:407] Connection header received
0806 03:46:53.904335 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:46:53.904336 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:46:53.904344 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:46:53.904383 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:46:53.904762 (+ 379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:46:53.905353 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:46:53.906471 (+ 1118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:46:53.906648 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:46:53.906701 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:46:53.907688 (+ 987us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:46:53.907689 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:46:53.907689 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:46:53.907695 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:46:53.909090 (+ 1395us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:46:53.909514 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:46:53.909514 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:46:53.909515 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:46:53.909564 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:46:53.909785 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:46:53.909785 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:46:53.909785 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:46:53.909982 (+ 197us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:46:53.910038 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0806 03:46:53.910213 (+ 175us) server_negotiation.cc:299] Negotiation successful
0806 03:46:53.910273 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260806 03:46:53.910425 15114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:46:53.904060 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:42173 (local address 127.11.253.194:47755)
0806 03:46:53.904200 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0806 03:46:53.904212 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 03:46:53.904276 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:46:53.904420 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:46:53.904421 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:46:53.904621 (+ 200us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:46:53.904681 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:46:53.904683 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:46:53.905543 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:46:53.905544 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:46:53.906393 (+ 849us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:46:53.906396 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:46:53.906729 (+ 333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:46:53.906730 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:46:53.906811 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:46:53.907191 (+ 380us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:46:53.907196 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:46:53.907601 (+ 405us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:46:53.909179 (+ 1578us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:46:53.909180 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:46:53.909180 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:46:53.909428 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:46:53.909630 (+ 202us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:46:53.909631 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:46:53.909631 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:46:53.909716 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:46:53.910051 (+ 335us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:46:53.910052 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:46:53.910105 (+ 53us) client_negotiation.cc:769] Sending connection context
0806 03:46:53.910187 (+ 82us) client_negotiation.cc:240] Negotiation successful
0806 03:46:53.910302 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":82,"threads_started":1}
WARNING: no policy specified for kudu/127.11.253.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.253.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.253.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.195.keytab.
Entry for principal kudu/127.11.253.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.195.keytab.
I20260806 03:46:53.933594 12279 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskToVWRe/build/release/bin/kudu
/tmp/dist-test-taskToVWRe/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785988004637995-12279-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785988004637995-12279-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785988004637995-12279-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785988004637995-12279-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.253.195:0
--local_ip_for_outbound_sockets=127.11.253.195
--webserver_interface=127.11.253.195
--webserver_port=0
--tserver_master_addrs=127.11.253.254:42173
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.195.keytab
--principal=kudu/127.11.253.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.253.212:35741
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785988004637995-12279-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785988004637995-12279-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785988004637995-12279-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:46:54.020339 15116 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:46:54.020577 15116 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:46:54.020606 15116 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:46:54.021759 15116 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:46:54.021803 15116 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:46:54.021826 15116 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260806 03:46:54.021840 15116 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 03:46:54.021859 15116 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.253.195
I20260806 03:46:54.023450 15116 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.253.212:35741
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785988004637995-12279-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785988004637995-12279-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.195.keytab
--principal=kudu/127.11.253.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.253.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785988004637995-12279-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.253.195
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.11.253.254:42173
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15116
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.253.195
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785988004637995-12279-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:46:54.023671 15116 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:46:54.023882 15116 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:46:54.029685 15128 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:46:54.029695 15122 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:46:54.029670 15125 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:46:54.029951 15116 server_base.cc:1034] running on GCE node
I20260806 03:46:54.030167 15116 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:46:54.030444 15116 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:46:54.031639 15116 hybrid_clock.cc:648] HybridClock initialized: now 1785988014031621 us; error 30 us; skew 500 ppm
Aug 06 03:46:54 dist-test-slave-6ntk krb5kdc[14550](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988014, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.253.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:46:54.034530 15116 init.cc:377] Logged in from keytab as kudu/127.11.253.195@KRBTEST.COM (short username kudu)
I20260806 03:46:54.035068 15116 webserver.cc:460] Webserver started at http://127.11.253.195:37587/ using document root <none> and password file <none>
I20260806 03:46:54.035358 15116 fs_manager.cc:359] Metadata directory not provided
I20260806 03:46:54.035434 15116 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:46:54.035583 15116 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:46:54.036890 15116 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785988004637995-12279-0/minicluster-data/ts-2/data/instance:
uuid: "18c08fd1a0d84a61ab7e6b98a82d3c11"
format_stamp: "Formatted at 2026-08-06 03:46:54 on dist-test-slave-6ntk"
server_key: "6d948b361bb379fc4529569cc40b5756"
server_key_iv: "1c7f4a456c535b1a088a223cd031f2f2"
server_key_version: "encryptionkey@0"
I20260806 03:46:54.037277 15116 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785988004637995-12279-0/minicluster-data/ts-2/wal/instance:
uuid: "18c08fd1a0d84a61ab7e6b98a82d3c11"
format_stamp: "Formatted at 2026-08-06 03:46:54 on dist-test-slave-6ntk"
server_key: "6d948b361bb379fc4529569cc40b5756"
server_key_iv: "1c7f4a456c535b1a088a223cd031f2f2"
server_key_version: "encryptionkey@0"
I20260806 03:46:54.038913 15116 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.004s sys 0.000s
I20260806 03:46:54.039820 15138 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:46:54.040050 15116 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 03:46:54.040165 15116 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785988004637995-12279-0/minicluster-data/ts-2/data,/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785988004637995-12279-0/minicluster-data/ts-2/wal
uuid: "18c08fd1a0d84a61ab7e6b98a82d3c11"
format_stamp: "Formatted at 2026-08-06 03:46:54 on dist-test-slave-6ntk"
server_key: "6d948b361bb379fc4529569cc40b5756"
server_key_iv: "1c7f4a456c535b1a088a223cd031f2f2"
server_key_version: "encryptionkey@0"
I20260806 03:46:54.040246 15116 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785988004637995-12279-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785988004637995-12279-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785988004637995-12279-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:46:54.048938 15116 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:46:54.051424 15116 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:46:54.051595 15116 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:46:54.052160 15116 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 03:46:54.052201 15116 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 03:46:54.052224 15116 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 03:46:54.052275 15116 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 03:46:54.059628 15116 rpc_server.cc:307] RPC server started. Bound to: 127.11.253.195:45903
I20260806 03:46:54.059870 15299 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.253.195:45903 every 8 connection(s)
Aug 06 03:46:54 dist-test-slave-6ntk krb5kdc[14550](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988014, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.253.195@KRBTEST.COM for kudu/127.11.253.254@KRBTEST.COM
I20260806 03:46:54.060057 15116 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785988004637995-12279-0/minicluster-data/ts-2/data/info.pb
I20260806 03:46:54.063482 14680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:46:54.052896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.253.195:43143 (local address 127.11.253.254:42173)
0806 03:46:54.052961 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0806 03:46:54.052964 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 03:46:54.053187 (+ 223us) server_negotiation.cc:407] Connection header received
0806 03:46:54.053792 (+ 605us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:46:54.053794 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:46:54.053811 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:46:54.053917 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:46:54.054865 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:46:54.055745 (+ 880us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:46:54.057339 (+ 1594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:46:54.057669 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:46:54.057761 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:46:54.060679 (+ 2918us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:46:54.060680 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:46:54.060681 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:46:54.060690 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:46:54.062321 (+ 1631us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:46:54.062687 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:46:54.062688 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:46:54.062688 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:46:54.062739 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:46:54.062951 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:46:54.062952 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:46:54.062952 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:46:54.063055 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:46:54.063106 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0806 03:46:54.063329 (+ 223us) server_negotiation.cc:299] Negotiation successful
0806 03:46:54.063418 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260806 03:46:54.063656 15150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:46:54.052901 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:42173 (local address 127.11.253.195:43143)
0806 03:46:54.053075 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0806 03:46:54.053111 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0806 03:46:54.053660 (+ 549us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:46:54.054024 (+ 364us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:46:54.054028 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:46:54.054443 (+ 415us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:46:54.054754 (+ 311us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:46:54.054761 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:46:54.055845 (+ 1084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:46:54.055846 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:46:54.057235 (+ 1389us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:46:54.057238 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:46:54.057774 (+ 536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:46:54.057775 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:46:54.057936 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:46:54.058649 (+ 713us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:46:54.058660 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:46:54.060572 (+ 1912us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:46:54.062399 (+ 1827us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:46:54.062405 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:46:54.062407 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:46:54.062620 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:46:54.062800 (+ 180us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:46:54.062801 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:46:54.062801 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:46:54.062895 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:46:54.063111 (+ 216us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:46:54.063116 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:46:54.063228 (+ 112us) client_negotiation.cc:769] Sending connection context
0806 03:46:54.063315 (+ 87us) client_negotiation.cc:240] Negotiation successful
0806 03:46:54.063432 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":94,"threads_started":1}
I20260806 03:46:54.064133 15301 heartbeater.cc:347] Connected to a master server at 127.11.253.254:42173
I20260806 03:46:54.064217 15301 heartbeater.cc:464] Registering TS with master...
I20260806 03:46:54.064409 15301 heartbeater.cc:511] Master 127.11.253.254:42173 requested a full tablet report, sending...
I20260806 03:46:54.064844 14606 ts_manager.cc:194] Registered new tserver with Master: 18c08fd1a0d84a61ab7e6b98a82d3c11 (127.11.253.195:45903)
I20260806 03:46:54.065229 14606 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.253.195@KRBTEST.COM'} at 127.11.253.195:43143
I20260806 03:46:54.069955 12279 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskToVWRe/build/release/bin/kudu as pid 15116
I20260806 03:46:54.070043 12279 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785988004637995-12279-0/minicluster-data/ts-2/wal/instance
I20260806 03:46:54.070236 12279 external_mini_cluster.cc:1428] Setting key 47bea11c319953d66f037cb6ee217d7c
I20260806 03:46:54.071050 12279 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 03:46:54.071416 15307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:46:54.065548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:42173 (local address 127.11.253.195:58483)
0806 03:46:54.065691 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0806 03:46:54.065703 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 03:46:54.065784 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:46:54.065949 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:46:54.065950 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:46:54.066134 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:46:54.066183 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:46:54.066185 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:46:54.066829 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:46:54.066830 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:46:54.067340 (+ 510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:46:54.067341 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:46:54.067712 (+ 371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:46:54.067713 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:46:54.067787 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:46:54.068245 (+ 458us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:46:54.068249 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:46:54.068529 (+ 280us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:46:54.070291 (+ 1762us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:46:54.070292 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:46:54.070292 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:46:54.070495 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:46:54.070712 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:46:54.070713 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:46:54.070713 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:46:54.070774 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:46:54.071041 (+ 267us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:46:54.071042 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:46:54.071090 (+ 48us) client_negotiation.cc:769] Sending connection context
0806 03:46:54.071181 (+ 91us) client_negotiation.cc:240] Negotiation successful
0806 03:46:54.071287 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":69,"threads_started":1}
I20260806 03:46:54.071447 14680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:46:54.065589 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.253.195:58483 (local address 127.11.253.254:42173)
0806 03:46:54.065636 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0806 03:46:54.065638 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:46:54.065770 (+ 132us) server_negotiation.cc:407] Connection header received
0806 03:46:54.065843 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:46:54.065844 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:46:54.065855 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:46:54.065942 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:46:54.066264 (+ 322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:46:54.066733 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:46:54.067427 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:46:54.067637 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:46:54.067712 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:46:54.068635 (+ 923us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:46:54.068637 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:46:54.068637 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:46:54.068644 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:46:54.070200 (+ 1556us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:46:54.070572 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:46:54.070573 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:46:54.070574 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:46:54.070635 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:46:54.070848 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:46:54.070849 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:46:54.070849 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:46:54.070971 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:46:54.071049 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0806 03:46:54.071215 (+ 166us) server_negotiation.cc:299] Negotiation successful
0806 03:46:54.071285 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
Aug 06 03:46:54 dist-test-slave-6ntk krb5kdc[14550](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988014, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 03:46:54 dist-test-slave-6ntk krb5kdc[14550](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988014, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.253.254@KRBTEST.COM
I20260806 03:46:54.095498 14680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:46:54.088397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60630 (local address 127.11.253.254:42173)
0806 03:46:54.088472 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0806 03:46:54.088474 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:46:54.088483 (+ 9us) server_negotiation.cc:407] Connection header received
0806 03:46:54.088509 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:46:54.088510 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:46:54.088519 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:46:54.088572 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:46:54.089150 (+ 578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:46:54.089656 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:46:54.090435 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:46:54.090622 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:46:54.090675 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:46:54.092948 (+ 2273us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:46:54.092949 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:46:54.092950 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:46:54.092956 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:46:54.094447 (+ 1491us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:46:54.094826 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:46:54.094827 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:46:54.094827 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:46:54.094884 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:46:54.095066 (+ 182us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:46:54.095067 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:46:54.095067 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:46:54.095174 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:46:54.095253 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0806 03:46:54.095344 (+ 91us) server_negotiation.cc:299] Negotiation successful
0806 03:46:54.095429 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260806 03:46:54.097259 14606 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60630:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 03:46:54.099020 14606 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 03:46:54.106140 15321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:46:54.101482 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.193:41343 (local address 127.0.0.1:55358)
0806 03:46:54.101656 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0806 03:46:54.101698 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 03:46:54.101770 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:46:54.102199 (+ 429us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:46:54.102200 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:46:54.102202 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 03:46:54.102267 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:46:54.102269 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:46:54.103515 (+ 1246us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:46:54.103516 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:46:54.105025 (+ 1509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:46:54.105027 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:46:54.105718 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:46:54.105719 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:46:54.105801 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:46:54.105815 (+ 14us) client_negotiation.cc:769] Sending connection context
0806 03:46:54.105969 (+ 154us) client_negotiation.cc:240] Negotiation successful
0806 03:46:54.106033 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"mutex_wait_us":59,"spinlock_wait_cycles":768,"thread_start_us":104,"threads_started":1}
I20260806 03:46:54.106225 15331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:46:54.101766 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35384 (local address 127.11.253.195:45903)
0806 03:46:54.102310 (+ 544us) server_negotiation.cc:206] Beginning negotiation
0806 03:46:54.102314 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 03:46:54.102324 (+ 10us) server_negotiation.cc:407] Connection header received
0806 03:46:54.102371 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:46:54.102373 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:46:54.102390 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:46:54.102463 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 03:46:54.102589 (+ 126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:46:54.103654 (+ 1065us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:46:54.104912 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:46:54.105687 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:46:54.105760 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:46:54.105825 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0806 03:46:54.106080 (+ 255us) server_negotiation.cc:299] Negotiation successful
0806 03:46:54.106120 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":501,"thread_start_us":85,"threads_started":1}
I20260806 03:46:54.106336 15323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:46:54.101482 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.194:45549 (local address 127.0.0.1:44504)
0806 03:46:54.101898 (+ 416us) negotiation.cc:107] Waiting for socket to connect
0806 03:46:54.101909 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 03:46:54.101958 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:46:54.102149 (+ 191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:46:54.102152 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:46:54.102154 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 03:46:54.102226 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:46:54.102228 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:46:54.103631 (+ 1403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:46:54.103631 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:46:54.105012 (+ 1381us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:46:54.105015 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:46:54.105949 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:46:54.105950 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:46:54.106063 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:46:54.106068 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 03:46:54.106119 (+ 51us) client_negotiation.cc:240] Negotiation successful
0806 03:46:54.106156 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":371,"mutex_wait_us":89,"thread_start_us":63,"threads_started":1}
I20260806 03:46:54.106230 15322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:46:54.101493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.195:45903 (local address 127.0.0.1:35384)
0806 03:46:54.101919 (+ 426us) negotiation.cc:107] Waiting for socket to connect
0806 03:46:54.101932 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 03:46:54.101994 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:46:54.102459 (+ 465us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:46:54.102459 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:46:54.102461 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 03:46:54.102509 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:46:54.102511 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:46:54.103734 (+ 1223us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:46:54.103735 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:46:54.104826 (+ 1091us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:46:54.104829 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:46:54.105906 (+ 1077us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:46:54.105908 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:46:54.105999 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:46:54.106006 (+ 7us) client_negotiation.cc:769] Sending connection context
0806 03:46:54.106062 (+ 56us) client_negotiation.cc:240] Negotiation successful
0806 03:46:54.106104 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":403,"thread_start_us":94,"threads_started":1}
I20260806 03:46:54.106500 15327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:46:54.101683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55358 (local address 127.11.253.193:41343)
0806 03:46:54.101997 (+ 314us) server_negotiation.cc:206] Beginning negotiation
0806 03:46:54.102002 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 03:46:54.102013 (+ 11us) server_negotiation.cc:407] Connection header received
0806 03:46:54.102055 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:46:54.102057 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:46:54.102072 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:46:54.102158 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 03:46:54.102591 (+ 433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:46:54.103427 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:46:54.105115 (+ 1688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:46:54.105644 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:46:54.105718 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:46:54.105779 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0806 03:46:54.106346 (+ 567us) server_negotiation.cc:299] Negotiation successful
0806 03:46:54.106398 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":275,"thread_start_us":105,"threads_started":1}
I20260806 03:46:54.107089 15326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:46:54.101697 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44504 (local address 127.11.253.194:45549)
0806 03:46:54.101972 (+ 275us) server_negotiation.cc:206] Beginning negotiation
0806 03:46:54.101977 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 03:46:54.101988 (+ 11us) server_negotiation.cc:407] Connection header received
0806 03:46:54.102027 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:46:54.102028 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:46:54.102051 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:46:54.102138 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 03:46:54.102326 (+ 188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:46:54.103550 (+ 1224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:46:54.105106 (+ 1556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:46:54.105811 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:46:54.105910 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:46:54.105978 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0806 03:46:54.106205 (+ 227us) server_negotiation.cc:299] Negotiation successful
0806 03:46:54.106988 (+ 783us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":241,"thread_start_us":92,"threads_started":1}
I20260806 03:46:54.107244 14801 tablet_service.cc:1467] Processing CreateTablet for tablet 6ba15e72b6e240b88769688991489b1f (DEFAULT_TABLE table=test-table [id=9048e77224d74044a4f4698d5f4edd85]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 03:46:54.107220 15204 tablet_service.cc:1467] Processing CreateTablet for tablet 6ba15e72b6e240b88769688991489b1f (DEFAULT_TABLE table=test-table [id=9048e77224d74044a4f4698d5f4edd85]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 03:46:54.107676 14801 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6ba15e72b6e240b88769688991489b1f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:46:54.107697 15204 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6ba15e72b6e240b88769688991489b1f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:46:54.110427 15002 tablet_service.cc:1467] Processing CreateTablet for tablet 6ba15e72b6e240b88769688991489b1f (DEFAULT_TABLE table=test-table [id=9048e77224d74044a4f4698d5f4edd85]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 03:46:54.110571 15332 tablet_bootstrap.cc:492] T 6ba15e72b6e240b88769688991489b1f P 18c08fd1a0d84a61ab7e6b98a82d3c11: Bootstrap starting.
I20260806 03:46:54.110818 15002 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6ba15e72b6e240b88769688991489b1f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:46:54.111186 15333 tablet_bootstrap.cc:492] T 6ba15e72b6e240b88769688991489b1f P 2577e879d33e41ffb5a214cdae767802: Bootstrap starting.
I20260806 03:46:54.111732 15332 tablet_bootstrap.cc:654] T 6ba15e72b6e240b88769688991489b1f P 18c08fd1a0d84a61ab7e6b98a82d3c11: Neither blocks nor log segments found. Creating new log.
I20260806 03:46:54.111990 15333 tablet_bootstrap.cc:654] T 6ba15e72b6e240b88769688991489b1f P 2577e879d33e41ffb5a214cdae767802: Neither blocks nor log segments found. Creating new log.
I20260806 03:46:54.112114 15332 log.cc:824] T 6ba15e72b6e240b88769688991489b1f P 18c08fd1a0d84a61ab7e6b98a82d3c11: Log is configured to *not* fsync() on all Append() calls
I20260806 03:46:54.112473 15333 log.cc:824] T 6ba15e72b6e240b88769688991489b1f P 2577e879d33e41ffb5a214cdae767802: Log is configured to *not* fsync() on all Append() calls
I20260806 03:46:54.113096 15332 tablet_bootstrap.cc:492] T 6ba15e72b6e240b88769688991489b1f P 18c08fd1a0d84a61ab7e6b98a82d3c11: No bootstrap required, opened a new log
I20260806 03:46:54.113291 15333 tablet_bootstrap.cc:492] T 6ba15e72b6e240b88769688991489b1f P 2577e879d33e41ffb5a214cdae767802: No bootstrap required, opened a new log
I20260806 03:46:54.113371 15333 ts_tablet_manager.cc:1397] T 6ba15e72b6e240b88769688991489b1f P 2577e879d33e41ffb5a214cdae767802: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 03:46:54.113322 15332 ts_tablet_manager.cc:1397] T 6ba15e72b6e240b88769688991489b1f P 18c08fd1a0d84a61ab7e6b98a82d3c11: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 03:46:54.114147 15341 tablet_bootstrap.cc:492] T 6ba15e72b6e240b88769688991489b1f P 74eb6a2589da4205b5e5623842ef6faa: Bootstrap starting.
I20260806 03:46:54.114732 15341 tablet_bootstrap.cc:654] T 6ba15e72b6e240b88769688991489b1f P 74eb6a2589da4205b5e5623842ef6faa: Neither blocks nor log segments found. Creating new log.
I20260806 03:46:54.114981 15341 log.cc:824] T 6ba15e72b6e240b88769688991489b1f P 74eb6a2589da4205b5e5623842ef6faa: Log is configured to *not* fsync() on all Append() calls
I20260806 03:46:54.115522 15333 raft_consensus.cc:348] T 6ba15e72b6e240b88769688991489b1f P 2577e879d33e41ffb5a214cdae767802 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2577e879d33e41ffb5a214cdae767802" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 41343 } } peers { permanent_uuid: "74eb6a2589da4205b5e5623842ef6faa" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 45549 } } peers { permanent_uuid: "18c08fd1a0d84a61ab7e6b98a82d3c11" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 45903 } }
I20260806 03:46:54.115667 15341 tablet_bootstrap.cc:492] T 6ba15e72b6e240b88769688991489b1f P 74eb6a2589da4205b5e5623842ef6faa: No bootstrap required, opened a new log
I20260806 03:46:54.115727 15341 ts_tablet_manager.cc:1397] T 6ba15e72b6e240b88769688991489b1f P 74eb6a2589da4205b5e5623842ef6faa: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 03:46:54.115713 15333 raft_consensus.cc:374] T 6ba15e72b6e240b88769688991489b1f P 2577e879d33e41ffb5a214cdae767802 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:46:54.115759 15333 raft_consensus.cc:729] T 6ba15e72b6e240b88769688991489b1f P 2577e879d33e41ffb5a214cdae767802 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2577e879d33e41ffb5a214cdae767802, State: Initialized, Role: FOLLOWER
I20260806 03:46:54.115880 15333 consensus_queue.cc:260] T 6ba15e72b6e240b88769688991489b1f P 2577e879d33e41ffb5a214cdae767802 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2577e879d33e41ffb5a214cdae767802" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 41343 } } peers { permanent_uuid: "74eb6a2589da4205b5e5623842ef6faa" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 45549 } } peers { permanent_uuid: "18c08fd1a0d84a61ab7e6b98a82d3c11" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 45903 } }
I20260806 03:46:54.116163 15333 ts_tablet_manager.cc:1428] T 6ba15e72b6e240b88769688991489b1f P 2577e879d33e41ffb5a214cdae767802: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 03:46:54.116278 14894 heartbeater.cc:503] Master 127.11.253.254:42173 was elected leader, sending a full tablet report...
I20260806 03:46:54.115653 15332 raft_consensus.cc:348] T 6ba15e72b6e240b88769688991489b1f P 18c08fd1a0d84a61ab7e6b98a82d3c11 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2577e879d33e41ffb5a214cdae767802" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 41343 } } peers { permanent_uuid: "74eb6a2589da4205b5e5623842ef6faa" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 45549 } } peers { permanent_uuid: "18c08fd1a0d84a61ab7e6b98a82d3c11" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 45903 } }
I20260806 03:46:54.116559 15332 raft_consensus.cc:374] T 6ba15e72b6e240b88769688991489b1f P 18c08fd1a0d84a61ab7e6b98a82d3c11 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:46:54.116626 15332 raft_consensus.cc:729] T 6ba15e72b6e240b88769688991489b1f P 18c08fd1a0d84a61ab7e6b98a82d3c11 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 18c08fd1a0d84a61ab7e6b98a82d3c11, State: Initialized, Role: FOLLOWER
I20260806 03:46:54.117111 15332 consensus_queue.cc:260] T 6ba15e72b6e240b88769688991489b1f P 18c08fd1a0d84a61ab7e6b98a82d3c11 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2577e879d33e41ffb5a214cdae767802" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 41343 } } peers { permanent_uuid: "74eb6a2589da4205b5e5623842ef6faa" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 45549 } } peers { permanent_uuid: "18c08fd1a0d84a61ab7e6b98a82d3c11" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 45903 } }
I20260806 03:46:54.117390 15332 ts_tablet_manager.cc:1428] T 6ba15e72b6e240b88769688991489b1f P 18c08fd1a0d84a61ab7e6b98a82d3c11: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260806 03:46:54.117470 15301 heartbeater.cc:503] Master 127.11.253.254:42173 was elected leader, sending a full tablet report...
I20260806 03:46:54.117893 15341 raft_consensus.cc:348] T 6ba15e72b6e240b88769688991489b1f P 74eb6a2589da4205b5e5623842ef6faa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2577e879d33e41ffb5a214cdae767802" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 41343 } } peers { permanent_uuid: "74eb6a2589da4205b5e5623842ef6faa" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 45549 } } peers { permanent_uuid: "18c08fd1a0d84a61ab7e6b98a82d3c11" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 45903 } }
I20260806 03:46:54.118068 15341 raft_consensus.cc:374] T 6ba15e72b6e240b88769688991489b1f P 74eb6a2589da4205b5e5623842ef6faa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:46:54.118117 15341 raft_consensus.cc:729] T 6ba15e72b6e240b88769688991489b1f P 74eb6a2589da4205b5e5623842ef6faa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74eb6a2589da4205b5e5623842ef6faa, State: Initialized, Role: FOLLOWER
I20260806 03:46:54.118229 15341 consensus_queue.cc:260] T 6ba15e72b6e240b88769688991489b1f P 74eb6a2589da4205b5e5623842ef6faa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2577e879d33e41ffb5a214cdae767802" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 41343 } } peers { permanent_uuid: "74eb6a2589da4205b5e5623842ef6faa" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 45549 } } peers { permanent_uuid: "18c08fd1a0d84a61ab7e6b98a82d3c11" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 45903 } }
I20260806 03:46:54.118498 15341 ts_tablet_manager.cc:1428] T 6ba15e72b6e240b88769688991489b1f P 74eb6a2589da4205b5e5623842ef6faa: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 03:46:54.118520 15102 heartbeater.cc:503] Master 127.11.253.254:42173 was elected leader, sending a full tablet report...
W20260806 03:46:54.149370 15103 tablet.cc:2369] T 6ba15e72b6e240b88769688991489b1f P 74eb6a2589da4205b5e5623842ef6faa: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 03:46:54.232620 14896 tablet.cc:2369] T 6ba15e72b6e240b88769688991489b1f P 2577e879d33e41ffb5a214cdae767802: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 03:46:54.257782 15345 raft_consensus.cc:482] T 6ba15e72b6e240b88769688991489b1f P 18c08fd1a0d84a61ab7e6b98a82d3c11 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 03:46:54.257975 15345 raft_consensus.cc:504] T 6ba15e72b6e240b88769688991489b1f P 18c08fd1a0d84a61ab7e6b98a82d3c11 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2577e879d33e41ffb5a214cdae767802" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 41343 } } peers { permanent_uuid: "74eb6a2589da4205b5e5623842ef6faa" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 45549 } } peers { permanent_uuid: "18c08fd1a0d84a61ab7e6b98a82d3c11" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 45903 } }
I20260806 03:46:54.258440 15345 leader_election.cc:288] T 6ba15e72b6e240b88769688991489b1f P 18c08fd1a0d84a61ab7e6b98a82d3c11 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2577e879d33e41ffb5a214cdae767802 (127.11.253.193:41343), 74eb6a2589da4205b5e5623842ef6faa (127.11.253.194:45549)
I20260806 03:46:54.262640 15326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:46:54.258745 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.253.195:55959 (local address 127.11.253.194:45549)
0806 03:46:54.258806 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0806 03:46:54.258808 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:46:54.258939 (+ 131us) server_negotiation.cc:407] Connection header received
0806 03:46:54.258980 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:46:54.258981 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:46:54.258991 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:46:54.259044 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 03:46:54.259229 (+ 185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:46:54.260003 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:46:54.261575 (+ 1572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:46:54.262175 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:46:54.262255 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:46:54.262265 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0806 03:46:54.262525 (+ 260us) server_negotiation.cc:299] Negotiation successful
0806 03:46:54.262568 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260806 03:46:54.262640 15327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:46:54.258797 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.253.195:59055 (local address 127.11.253.193:41343)
0806 03:46:54.258852 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0806 03:46:54.258856 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 03:46:54.258869 (+ 13us) server_negotiation.cc:407] Connection header received
0806 03:46:54.258894 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:46:54.258895 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:46:54.258902 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:46:54.258983 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 03:46:54.259225 (+ 242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:46:54.260007 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:46:54.261578 (+ 1571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:46:54.262175 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:46:54.262255 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:46:54.262277 (+ 22us) server_negotiation.cc:1035] Waiting for connection context
0806 03:46:54.262525 (+ 248us) server_negotiation.cc:299] Negotiation successful
0806 03:46:54.262568 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260806 03:46:54.262642 15150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:46:54.258624 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.193:41343 (local address 127.11.253.195:59055)
0806 03:46:54.258700 (+ 76us) negotiation.cc:107] Waiting for socket to connect
0806 03:46:54.258715 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 03:46:54.258763 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:46:54.259036 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:46:54.259037 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:46:54.259039 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 03:46:54.259120 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:46:54.259122 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:46:54.260108 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:46:54.260109 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:46:54.261478 (+ 1369us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:46:54.261481 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:46:54.262366 (+ 885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:46:54.262366 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:46:54.262450 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:46:54.262452 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 03:46:54.262509 (+ 57us) client_negotiation.cc:240] Negotiation successful
0806 03:46:54.262539 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":54,"mutex_wait_us":44}
I20260806 03:46:54.262656 15348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:46:54.258662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.194:45549 (local address 127.11.253.195:55959)
0806 03:46:54.258855 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0806 03:46:54.258870 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 03:46:54.258926 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:46:54.259059 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:46:54.259059 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:46:54.259060 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 03:46:54.259153 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:46:54.259155 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:46:54.260108 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:46:54.260109 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:46:54.261478 (+ 1369us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:46:54.261481 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:46:54.262326 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:46:54.262328 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:46:54.262413 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:46:54.262418 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 03:46:54.262494 (+ 76us) client_negotiation.cc:240] Negotiation successful
0806 03:46:54.262539 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":171,"mutex_wait_us":147,"thread_start_us":100,"threads_started":1}
I20260806 03:46:54.263124 15037 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ba15e72b6e240b88769688991489b1f" candidate_uuid: "18c08fd1a0d84a61ab7e6b98a82d3c11" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74eb6a2589da4205b5e5623842ef6faa" is_pre_election: true
I20260806 03:46:54.263120 14837 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ba15e72b6e240b88769688991489b1f" candidate_uuid: "18c08fd1a0d84a61ab7e6b98a82d3c11" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2577e879d33e41ffb5a214cdae767802" is_pre_election: true
I20260806 03:46:54.263299 15037 raft_consensus.cc:2454] T 6ba15e72b6e240b88769688991489b1f P 74eb6a2589da4205b5e5623842ef6faa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 18c08fd1a0d84a61ab7e6b98a82d3c11 in term 0.
I20260806 03:46:54.263314 14837 raft_consensus.cc:2454] T 6ba15e72b6e240b88769688991489b1f P 2577e879d33e41ffb5a214cdae767802 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 18c08fd1a0d84a61ab7e6b98a82d3c11 in term 0.
I20260806 03:46:54.263545 15141 leader_election.cc:302] T 6ba15e72b6e240b88769688991489b1f P 18c08fd1a0d84a61ab7e6b98a82d3c11 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 18c08fd1a0d84a61ab7e6b98a82d3c11, 74eb6a2589da4205b5e5623842ef6faa; no voters:
I20260806 03:46:54.263707 15345 raft_consensus.cc:2781] T 6ba15e72b6e240b88769688991489b1f P 18c08fd1a0d84a61ab7e6b98a82d3c11 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 03:46:54.263794 15345 raft_consensus.cc:482] T 6ba15e72b6e240b88769688991489b1f P 18c08fd1a0d84a61ab7e6b98a82d3c11 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 03:46:54.263831 15345 raft_consensus.cc:3037] T 6ba15e72b6e240b88769688991489b1f P 18c08fd1a0d84a61ab7e6b98a82d3c11 [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:46:54.264648 15345 raft_consensus.cc:504] T 6ba15e72b6e240b88769688991489b1f P 18c08fd1a0d84a61ab7e6b98a82d3c11 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2577e879d33e41ffb5a214cdae767802" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 41343 } } peers { permanent_uuid: "74eb6a2589da4205b5e5623842ef6faa" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 45549 } } peers { permanent_uuid: "18c08fd1a0d84a61ab7e6b98a82d3c11" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 45903 } }
I20260806 03:46:54.264808 15345 leader_election.cc:288] T 6ba15e72b6e240b88769688991489b1f P 18c08fd1a0d84a61ab7e6b98a82d3c11 [CANDIDATE]: Term 1 election: Requested vote from peers 2577e879d33e41ffb5a214cdae767802 (127.11.253.193:41343), 74eb6a2589da4205b5e5623842ef6faa (127.11.253.194:45549)
I20260806 03:46:54.264959 14837 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ba15e72b6e240b88769688991489b1f" candidate_uuid: "18c08fd1a0d84a61ab7e6b98a82d3c11" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2577e879d33e41ffb5a214cdae767802"
I20260806 03:46:54.265044 14837 raft_consensus.cc:3037] T 6ba15e72b6e240b88769688991489b1f P 2577e879d33e41ffb5a214cdae767802 [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:46:54.265834 14837 raft_consensus.cc:2454] T 6ba15e72b6e240b88769688991489b1f P 2577e879d33e41ffb5a214cdae767802 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 18c08fd1a0d84a61ab7e6b98a82d3c11 in term 1.
I20260806 03:46:54.265976 15037 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ba15e72b6e240b88769688991489b1f" candidate_uuid: "18c08fd1a0d84a61ab7e6b98a82d3c11" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74eb6a2589da4205b5e5623842ef6faa"
I20260806 03:46:54.266069 15037 raft_consensus.cc:3037] T 6ba15e72b6e240b88769688991489b1f P 74eb6a2589da4205b5e5623842ef6faa [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:46:54.266482 15142 leader_election.cc:302] T 6ba15e72b6e240b88769688991489b1f P 18c08fd1a0d84a61ab7e6b98a82d3c11 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 18c08fd1a0d84a61ab7e6b98a82d3c11, 2577e879d33e41ffb5a214cdae767802; no voters:
I20260806 03:46:54.266630 15345 raft_consensus.cc:2781] T 6ba15e72b6e240b88769688991489b1f P 18c08fd1a0d84a61ab7e6b98a82d3c11 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 03:46:54.266834 15345 raft_consensus.cc:686] T 6ba15e72b6e240b88769688991489b1f P 18c08fd1a0d84a61ab7e6b98a82d3c11 [term 1 LEADER]: Becoming Leader. State: Replica: 18c08fd1a0d84a61ab7e6b98a82d3c11, State: Running, Role: LEADER
I20260806 03:46:54.266949 15345 consensus_queue.cc:237] T 6ba15e72b6e240b88769688991489b1f P 18c08fd1a0d84a61ab7e6b98a82d3c11 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2577e879d33e41ffb5a214cdae767802" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 41343 } } peers { permanent_uuid: "74eb6a2589da4205b5e5623842ef6faa" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 45549 } } peers { permanent_uuid: "18c08fd1a0d84a61ab7e6b98a82d3c11" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 45903 } }
I20260806 03:46:54.267009 15037 raft_consensus.cc:2454] T 6ba15e72b6e240b88769688991489b1f P 74eb6a2589da4205b5e5623842ef6faa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 18c08fd1a0d84a61ab7e6b98a82d3c11 in term 1.
I20260806 03:46:54.268008 14606 catalog_manager.cc:5495] T 6ba15e72b6e240b88769688991489b1f P 18c08fd1a0d84a61ab7e6b98a82d3c11 reported cstate change: term changed from 0 to 1, leader changed from <none> to 18c08fd1a0d84a61ab7e6b98a82d3c11 (127.11.253.195). New cstate: current_term: 1 leader_uuid: "18c08fd1a0d84a61ab7e6b98a82d3c11" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2577e879d33e41ffb5a214cdae767802" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 41343 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "74eb6a2589da4205b5e5623842ef6faa" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 45549 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "18c08fd1a0d84a61ab7e6b98a82d3c11" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 45903 } health_report { overall_health: HEALTHY } } }
Aug 06 03:46:54 dist-test-slave-6ntk krb5kdc[14550](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988013, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.253.254@KRBTEST.COM for kudu/127.11.253.254@KRBTEST.COM
I20260806 03:46:54.310600 15357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:46:54.302796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:42173 (local address 127.0.0.1:60642)
0806 03:46:54.302998 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0806 03:46:54.303009 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 03:46:54.303079 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:46:54.303206 (+ 127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:46:54.303207 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:46:54.303659 (+ 452us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:46:54.303710 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:46:54.303713 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:46:54.304361 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:46:54.304362 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:46:54.304897 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:46:54.304898 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:46:54.305376 (+ 478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:46:54.305378 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:46:54.305448 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:46:54.306116 (+ 668us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:46:54.306121 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:46:54.307705 (+ 1584us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:46:54.309365 (+ 1660us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:46:54.309384 (+ 19us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:46:54.309386 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:46:54.309629 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:46:54.309855 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:46:54.309856 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:46:54.309856 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:46:54.310019 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:46:54.310251 (+ 232us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:46:54.310257 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:46:54.310341 (+ 84us) client_negotiation.cc:769] Sending connection context
0806 03:46:54.310398 (+ 57us) client_negotiation.cc:240] Negotiation successful
0806 03:46:54.310499 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":124,"threads_started":1}
I20260806 03:46:54.310689 14680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:46:54.302840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60642 (local address 127.11.253.254:42173)
0806 03:46:54.302920 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0806 03:46:54.302923 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 03:46:54.303080 (+ 157us) server_negotiation.cc:407] Connection header received
0806 03:46:54.303123 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:46:54.303124 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:46:54.303134 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:46:54.303188 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:46:54.303780 (+ 592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:46:54.304289 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:46:54.304968 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:46:54.305218 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:46:54.305272 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:46:54.307807 (+ 2535us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:46:54.307808 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:46:54.307808 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:46:54.307815 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:46:54.309267 (+ 1452us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:46:54.309722 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:46:54.309723 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:46:54.309723 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:46:54.309773 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:46:54.310082 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:46:54.310082 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:46:54.310083 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:46:54.310187 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:46:54.310243 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0806 03:46:54.310424 (+ 181us) server_negotiation.cc:299] Negotiation successful
0806 03:46:54.310548 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":54,"server-negotiator.queue_time_us":48}
W20260806 03:46:54.310806 15302 tablet.cc:2369] T 6ba15e72b6e240b88769688991489b1f P 18c08fd1a0d84a61ab7e6b98a82d3c11: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 03:46:54.312904 14606 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.11.253.254@KRBTEST.COM'} at 127.0.0.1:60642:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260806 03:46:54.313088 14606 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 03:46:54.314922 14801 tablet_service.cc:1467] Processing CreateTablet for tablet fb0389a88b8a4fcb84a140efc6e5a09b (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=95b439f981794600a8bad3fc10444af7]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 03:46:54.314956 15002 tablet_service.cc:1467] Processing CreateTablet for tablet fb0389a88b8a4fcb84a140efc6e5a09b (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=95b439f981794600a8bad3fc10444af7]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 03:46:54.314976 15204 tablet_service.cc:1467] Processing CreateTablet for tablet fb0389a88b8a4fcb84a140efc6e5a09b (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=95b439f981794600a8bad3fc10444af7]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 03:46:54.315107 15002 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fb0389a88b8a4fcb84a140efc6e5a09b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:46:54.315132 15204 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fb0389a88b8a4fcb84a140efc6e5a09b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:46:54.315133 14801 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fb0389a88b8a4fcb84a140efc6e5a09b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:46:54.316552 15341 tablet_bootstrap.cc:492] T fb0389a88b8a4fcb84a140efc6e5a09b P 74eb6a2589da4205b5e5623842ef6faa: Bootstrap starting.
I20260806 03:46:54.317148 15332 tablet_bootstrap.cc:492] T fb0389a88b8a4fcb84a140efc6e5a09b P 18c08fd1a0d84a61ab7e6b98a82d3c11: Bootstrap starting.
I20260806 03:46:54.317147 15333 tablet_bootstrap.cc:492] T fb0389a88b8a4fcb84a140efc6e5a09b P 2577e879d33e41ffb5a214cdae767802: Bootstrap starting.
I20260806 03:46:54.317257 15341 tablet_bootstrap.cc:654] T fb0389a88b8a4fcb84a140efc6e5a09b P 74eb6a2589da4205b5e5623842ef6faa: Neither blocks nor log segments found. Creating new log.
I20260806 03:46:54.317581 15333 tablet_bootstrap.cc:654] T fb0389a88b8a4fcb84a140efc6e5a09b P 2577e879d33e41ffb5a214cdae767802: Neither blocks nor log segments found. Creating new log.
I20260806 03:46:54.317602 15332 tablet_bootstrap.cc:654] T fb0389a88b8a4fcb84a140efc6e5a09b P 18c08fd1a0d84a61ab7e6b98a82d3c11: Neither blocks nor log segments found. Creating new log.
I20260806 03:46:54.318300 15341 tablet_bootstrap.cc:492] T fb0389a88b8a4fcb84a140efc6e5a09b P 74eb6a2589da4205b5e5623842ef6faa: No bootstrap required, opened a new log
I20260806 03:46:54.318360 15341 ts_tablet_manager.cc:1397] T fb0389a88b8a4fcb84a140efc6e5a09b P 74eb6a2589da4205b5e5623842ef6faa: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 03:46:54.318406 15332 tablet_bootstrap.cc:492] T fb0389a88b8a4fcb84a140efc6e5a09b P 18c08fd1a0d84a61ab7e6b98a82d3c11: No bootstrap required, opened a new log
I20260806 03:46:54.318468 15332 ts_tablet_manager.cc:1397] T fb0389a88b8a4fcb84a140efc6e5a09b P 18c08fd1a0d84a61ab7e6b98a82d3c11: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260806 03:46:54.318512 15341 raft_consensus.cc:348] T fb0389a88b8a4fcb84a140efc6e5a09b P 74eb6a2589da4205b5e5623842ef6faa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2577e879d33e41ffb5a214cdae767802" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 41343 } } peers { permanent_uuid: "74eb6a2589da4205b5e5623842ef6faa" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 45549 } } peers { permanent_uuid: "18c08fd1a0d84a61ab7e6b98a82d3c11" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 45903 } }
I20260806 03:46:54.318586 15341 raft_consensus.cc:374] T fb0389a88b8a4fcb84a140efc6e5a09b P 74eb6a2589da4205b5e5623842ef6faa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:46:54.318670 15341 raft_consensus.cc:729] T fb0389a88b8a4fcb84a140efc6e5a09b P 74eb6a2589da4205b5e5623842ef6faa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74eb6a2589da4205b5e5623842ef6faa, State: Initialized, Role: FOLLOWER
I20260806 03:46:54.318760 15341 consensus_queue.cc:260] T fb0389a88b8a4fcb84a140efc6e5a09b P 74eb6a2589da4205b5e5623842ef6faa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2577e879d33e41ffb5a214cdae767802" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 41343 } } peers { permanent_uuid: "74eb6a2589da4205b5e5623842ef6faa" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 45549 } } peers { permanent_uuid: "18c08fd1a0d84a61ab7e6b98a82d3c11" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 45903 } }
I20260806 03:46:54.318677 15332 raft_consensus.cc:348] T fb0389a88b8a4fcb84a140efc6e5a09b P 18c08fd1a0d84a61ab7e6b98a82d3c11 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2577e879d33e41ffb5a214cdae767802" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 41343 } } peers { permanent_uuid: "74eb6a2589da4205b5e5623842ef6faa" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 45549 } } peers { permanent_uuid: "18c08fd1a0d84a61ab7e6b98a82d3c11" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 45903 } }
I20260806 03:46:54.318819 15332 raft_consensus.cc:374] T fb0389a88b8a4fcb84a140efc6e5a09b P 18c08fd1a0d84a61ab7e6b98a82d3c11 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:46:54.318851 15332 raft_consensus.cc:729] T fb0389a88b8a4fcb84a140efc6e5a09b P 18c08fd1a0d84a61ab7e6b98a82d3c11 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 18c08fd1a0d84a61ab7e6b98a82d3c11, State: Initialized, Role: FOLLOWER
I20260806 03:46:54.318873 15341 ts_tablet_manager.cc:1428] T fb0389a88b8a4fcb84a140efc6e5a09b P 74eb6a2589da4205b5e5623842ef6faa: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 03:46:54.318912 15332 consensus_queue.cc:260] T fb0389a88b8a4fcb84a140efc6e5a09b P 18c08fd1a0d84a61ab7e6b98a82d3c11 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2577e879d33e41ffb5a214cdae767802" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 41343 } } peers { permanent_uuid: "74eb6a2589da4205b5e5623842ef6faa" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 45549 } } peers { permanent_uuid: "18c08fd1a0d84a61ab7e6b98a82d3c11" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 45903 } }
I20260806 03:46:54.318953 15347 tablet_replica.cc:439] T fb0389a88b8a4fcb84a140efc6e5a09b P 74eb6a2589da4205b5e5623842ef6faa: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2577e879d33e41ffb5a214cdae767802" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 41343 } } peers { permanent_uuid: "74eb6a2589da4205b5e5623842ef6faa" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 45549 } } peers { permanent_uuid: "18c08fd1a0d84a61ab7e6b98a82d3c11" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 45903 } } }
I20260806 03:46:54.319020 15332 ts_tablet_manager.cc:1428] T fb0389a88b8a4fcb84a140efc6e5a09b P 18c08fd1a0d84a61ab7e6b98a82d3c11: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260806 03:46:54.319029 15347 tablet_replica.cc:442] T fb0389a88b8a4fcb84a140efc6e5a09b P 74eb6a2589da4205b5e5623842ef6faa: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 03:46:54.319077 15345 tablet_replica.cc:439] T fb0389a88b8a4fcb84a140efc6e5a09b P 18c08fd1a0d84a61ab7e6b98a82d3c11: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2577e879d33e41ffb5a214cdae767802" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 41343 } } peers { permanent_uuid: "74eb6a2589da4205b5e5623842ef6faa" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 45549 } } peers { permanent_uuid: "18c08fd1a0d84a61ab7e6b98a82d3c11" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 45903 } } }
I20260806 03:46:54.319160 15345 tablet_replica.cc:442] T fb0389a88b8a4fcb84a140efc6e5a09b P 18c08fd1a0d84a61ab7e6b98a82d3c11: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 03:46:54.319300 15333 tablet_bootstrap.cc:492] T fb0389a88b8a4fcb84a140efc6e5a09b P 2577e879d33e41ffb5a214cdae767802: No bootstrap required, opened a new log
I20260806 03:46:54.319371 15333 ts_tablet_manager.cc:1397] T fb0389a88b8a4fcb84a140efc6e5a09b P 2577e879d33e41ffb5a214cdae767802: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 03:46:54.319561 15333 raft_consensus.cc:348] T fb0389a88b8a4fcb84a140efc6e5a09b P 2577e879d33e41ffb5a214cdae767802 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2577e879d33e41ffb5a214cdae767802" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 41343 } } peers { permanent_uuid: "74eb6a2589da4205b5e5623842ef6faa" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 45549 } } peers { permanent_uuid: "18c08fd1a0d84a61ab7e6b98a82d3c11" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 45903 } }
I20260806 03:46:54.319667 15333 raft_consensus.cc:374] T fb0389a88b8a4fcb84a140efc6e5a09b P 2577e879d33e41ffb5a214cdae767802 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:46:54.319706 15333 raft_consensus.cc:729] T fb0389a88b8a4fcb84a140efc6e5a09b P 2577e879d33e41ffb5a214cdae767802 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2577e879d33e41ffb5a214cdae767802, State: Initialized, Role: FOLLOWER
I20260806 03:46:54.319772 15333 consensus_queue.cc:260] T fb0389a88b8a4fcb84a140efc6e5a09b P 2577e879d33e41ffb5a214cdae767802 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2577e879d33e41ffb5a214cdae767802" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 41343 } } peers { permanent_uuid: "74eb6a2589da4205b5e5623842ef6faa" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 45549 } } peers { permanent_uuid: "18c08fd1a0d84a61ab7e6b98a82d3c11" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 45903 } }
I20260806 03:46:54.319916 15333 ts_tablet_manager.cc:1428] T fb0389a88b8a4fcb84a140efc6e5a09b P 2577e879d33e41ffb5a214cdae767802: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 03:46:54.319979 15343 tablet_replica.cc:439] T fb0389a88b8a4fcb84a140efc6e5a09b P 2577e879d33e41ffb5a214cdae767802: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2577e879d33e41ffb5a214cdae767802" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 41343 } } peers { permanent_uuid: "74eb6a2589da4205b5e5623842ef6faa" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 45549 } } peers { permanent_uuid: "18c08fd1a0d84a61ab7e6b98a82d3c11" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 45903 } } }
I20260806 03:46:54.320082 15343 tablet_replica.cc:442] T fb0389a88b8a4fcb84a140efc6e5a09b P 2577e879d33e41ffb5a214cdae767802: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 03:46:54.520656 15347 raft_consensus.cc:482] T fb0389a88b8a4fcb84a140efc6e5a09b P 74eb6a2589da4205b5e5623842ef6faa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 03:46:54.520771 15347 raft_consensus.cc:504] T fb0389a88b8a4fcb84a140efc6e5a09b P 74eb6a2589da4205b5e5623842ef6faa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2577e879d33e41ffb5a214cdae767802" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 41343 } } peers { permanent_uuid: "74eb6a2589da4205b5e5623842ef6faa" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 45549 } } peers { permanent_uuid: "18c08fd1a0d84a61ab7e6b98a82d3c11" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 45903 } }
I20260806 03:46:54.521240 15347 leader_election.cc:288] T fb0389a88b8a4fcb84a140efc6e5a09b P 74eb6a2589da4205b5e5623842ef6faa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2577e879d33e41ffb5a214cdae767802 (127.11.253.193:41343), 18c08fd1a0d84a61ab7e6b98a82d3c11 (127.11.253.195:45903)
I20260806 03:46:54.524253 15327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:46:54.521500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.253.194:50873 (local address 127.11.253.193:41343)
0806 03:46:54.521553 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0806 03:46:54.521556 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 03:46:54.521669 (+ 113us) server_negotiation.cc:407] Connection header received
0806 03:46:54.521754 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:46:54.521755 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:46:54.521763 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:46:54.521839 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 03:46:54.522068 (+ 229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:46:54.522532 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:46:54.523511 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:46:54.523888 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:46:54.523945 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:46:54.523953 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0806 03:46:54.524134 (+ 181us) server_negotiation.cc:299] Negotiation successful
0806 03:46:54.524178 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260806 03:46:54.524266 15365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:46:54.521417 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.193:41343 (local address 127.11.253.194:50873)
0806 03:46:54.521562 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0806 03:46:54.521573 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 03:46:54.521666 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:46:54.521861 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:46:54.521862 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:46:54.521890 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 03:46:54.521981 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:46:54.521984 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:46:54.522637 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:46:54.522638 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:46:54.523417 (+ 779us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:46:54.523419 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:46:54.523974 (+ 555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:46:54.523975 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:46:54.524049 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:46:54.524053 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 03:46:54.524106 (+ 53us) client_negotiation.cc:240] Negotiation successful
0806 03:46:54.524165 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":76,"threads_started":1}
I20260806 03:46:54.524292 15364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:46:54.521445 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.195:45903 (local address 127.11.253.194:49043)
0806 03:46:54.521576 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0806 03:46:54.521587 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 03:46:54.521643 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:46:54.521821 (+ 178us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:46:54.521822 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:46:54.521825 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 03:46:54.521936 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:46:54.521938 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:46:54.522626 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:46:54.522627 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:46:54.523430 (+ 803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:46:54.523432 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:46:54.524023 (+ 591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:46:54.524023 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:46:54.524077 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:46:54.524080 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 03:46:54.524158 (+ 78us) client_negotiation.cc:240] Negotiation successful
0806 03:46:54.524198 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"mutex_wait_us":51,"thread_start_us":54,"threads_started":1}
I20260806 03:46:54.524291 15331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:46:54.521600 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.253.194:49043 (local address 127.11.253.195:45903)
0806 03:46:54.521682 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0806 03:46:54.521684 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:46:54.521694 (+ 10us) server_negotiation.cc:407] Connection header received
0806 03:46:54.521725 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:46:54.521726 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:46:54.521737 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:46:54.521794 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 03:46:54.522009 (+ 215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:46:54.522519 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:46:54.523557 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:46:54.523911 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:46:54.524009 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:46:54.524016 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0806 03:46:54.524177 (+ 161us) server_negotiation.cc:299] Negotiation successful
0806 03:46:54.524214 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260806 03:46:54.524562 14837 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fb0389a88b8a4fcb84a140efc6e5a09b" candidate_uuid: "74eb6a2589da4205b5e5623842ef6faa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2577e879d33e41ffb5a214cdae767802" is_pre_election: true
I20260806 03:46:54.524673 14837 raft_consensus.cc:2454] T fb0389a88b8a4fcb84a140efc6e5a09b P 2577e879d33e41ffb5a214cdae767802 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74eb6a2589da4205b5e5623842ef6faa in term 0.
I20260806 03:46:54.524755 15236 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fb0389a88b8a4fcb84a140efc6e5a09b" candidate_uuid: "74eb6a2589da4205b5e5623842ef6faa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "18c08fd1a0d84a61ab7e6b98a82d3c11" is_pre_election: true
I20260806 03:46:54.524881 15236 raft_consensus.cc:2454] T fb0389a88b8a4fcb84a140efc6e5a09b P 18c08fd1a0d84a61ab7e6b98a82d3c11 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74eb6a2589da4205b5e5623842ef6faa in term 0.
I20260806 03:46:54.524868 14938 leader_election.cc:302] T fb0389a88b8a4fcb84a140efc6e5a09b P 74eb6a2589da4205b5e5623842ef6faa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2577e879d33e41ffb5a214cdae767802, 74eb6a2589da4205b5e5623842ef6faa; no voters:
I20260806 03:46:54.525004 15347 raft_consensus.cc:2781] T fb0389a88b8a4fcb84a140efc6e5a09b P 74eb6a2589da4205b5e5623842ef6faa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 03:46:54.525046 15347 raft_consensus.cc:482] T fb0389a88b8a4fcb84a140efc6e5a09b P 74eb6a2589da4205b5e5623842ef6faa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 03:46:54.525067 15347 raft_consensus.cc:3037] T fb0389a88b8a4fcb84a140efc6e5a09b P 74eb6a2589da4205b5e5623842ef6faa [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:46:54.525734 15347 raft_consensus.cc:504] T fb0389a88b8a4fcb84a140efc6e5a09b P 74eb6a2589da4205b5e5623842ef6faa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2577e879d33e41ffb5a214cdae767802" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 41343 } } peers { permanent_uuid: "74eb6a2589da4205b5e5623842ef6faa" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 45549 } } peers { permanent_uuid: "18c08fd1a0d84a61ab7e6b98a82d3c11" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 45903 } }
I20260806 03:46:54.526013 15347 leader_election.cc:288] T fb0389a88b8a4fcb84a140efc6e5a09b P 74eb6a2589da4205b5e5623842ef6faa [CANDIDATE]: Term 1 election: Requested vote from peers 2577e879d33e41ffb5a214cdae767802 (127.11.253.193:41343), 18c08fd1a0d84a61ab7e6b98a82d3c11 (127.11.253.195:45903)
I20260806 03:46:54.526140 15236 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fb0389a88b8a4fcb84a140efc6e5a09b" candidate_uuid: "74eb6a2589da4205b5e5623842ef6faa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "18c08fd1a0d84a61ab7e6b98a82d3c11"
I20260806 03:46:54.526147 14837 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fb0389a88b8a4fcb84a140efc6e5a09b" candidate_uuid: "74eb6a2589da4205b5e5623842ef6faa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2577e879d33e41ffb5a214cdae767802"
I20260806 03:46:54.526232 14837 raft_consensus.cc:3037] T fb0389a88b8a4fcb84a140efc6e5a09b P 2577e879d33e41ffb5a214cdae767802 [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:46:54.526235 15236 raft_consensus.cc:3037] T fb0389a88b8a4fcb84a140efc6e5a09b P 18c08fd1a0d84a61ab7e6b98a82d3c11 [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:46:54.526976 14837 raft_consensus.cc:2454] T fb0389a88b8a4fcb84a140efc6e5a09b P 2577e879d33e41ffb5a214cdae767802 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74eb6a2589da4205b5e5623842ef6faa in term 1.
I20260806 03:46:54.527066 15236 raft_consensus.cc:2454] T fb0389a88b8a4fcb84a140efc6e5a09b P 18c08fd1a0d84a61ab7e6b98a82d3c11 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74eb6a2589da4205b5e5623842ef6faa in term 1.
I20260806 03:46:54.527136 14938 leader_election.cc:302] T fb0389a88b8a4fcb84a140efc6e5a09b P 74eb6a2589da4205b5e5623842ef6faa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2577e879d33e41ffb5a214cdae767802, 74eb6a2589da4205b5e5623842ef6faa; no voters:
I20260806 03:46:54.527272 15347 raft_consensus.cc:2781] T fb0389a88b8a4fcb84a140efc6e5a09b P 74eb6a2589da4205b5e5623842ef6faa [term 1 FOLLOWER]: Leader election won for term 1
I20260806 03:46:54.527416 15347 raft_consensus.cc:686] T fb0389a88b8a4fcb84a140efc6e5a09b P 74eb6a2589da4205b5e5623842ef6faa [term 1 LEADER]: Becoming Leader. State: Replica: 74eb6a2589da4205b5e5623842ef6faa, State: Running, Role: LEADER
I20260806 03:46:54.527518 15347 consensus_queue.cc:237] T fb0389a88b8a4fcb84a140efc6e5a09b P 74eb6a2589da4205b5e5623842ef6faa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2577e879d33e41ffb5a214cdae767802" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 41343 } } peers { permanent_uuid: "74eb6a2589da4205b5e5623842ef6faa" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 45549 } } peers { permanent_uuid: "18c08fd1a0d84a61ab7e6b98a82d3c11" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 45903 } }
I20260806 03:46:54.527947 15366 tablet_replica.cc:439] T fb0389a88b8a4fcb84a140efc6e5a09b P 74eb6a2589da4205b5e5623842ef6faa: TxnStatusTablet state changed. Reason: New leader 74eb6a2589da4205b5e5623842ef6faa. Latest consensus state: current_term: 1 leader_uuid: "74eb6a2589da4205b5e5623842ef6faa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2577e879d33e41ffb5a214cdae767802" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 41343 } } peers { permanent_uuid: "74eb6a2589da4205b5e5623842ef6faa" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 45549 } } peers { permanent_uuid: "18c08fd1a0d84a61ab7e6b98a82d3c11" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 45903 } } }
I20260806 03:46:54.528043 15366 tablet_replica.cc:442] T fb0389a88b8a4fcb84a140efc6e5a09b P 74eb6a2589da4205b5e5623842ef6faa: This TxnStatusTablet replica's current role is: LEADER
I20260806 03:46:54.528304 15368 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260806 03:46:54.528442 15368 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260806 03:46:54.528396 14606 catalog_manager.cc:5495] T fb0389a88b8a4fcb84a140efc6e5a09b P 74eb6a2589da4205b5e5623842ef6faa reported cstate change: term changed from 0 to 1, leader changed from <none> to 74eb6a2589da4205b5e5623842ef6faa (127.11.253.194). New cstate: current_term: 1 leader_uuid: "74eb6a2589da4205b5e5623842ef6faa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2577e879d33e41ffb5a214cdae767802" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 41343 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "74eb6a2589da4205b5e5623842ef6faa" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 45549 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "18c08fd1a0d84a61ab7e6b98a82d3c11" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 45903 } health_report { overall_health: UNKNOWN } } }
I20260806 03:46:54.550195 15357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:46:54.547436 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.194:45549 (local address 127.0.0.1:44506)
0806 03:46:54.547552 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0806 03:46:54.547565 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 03:46:54.547615 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:46:54.547840 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:46:54.547841 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:46:54.547843 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 03:46:54.547898 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:46:54.547900 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:46:54.548601 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:46:54.548603 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:46:54.549184 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:46:54.549186 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:46:54.549566 (+ 380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:46:54.549567 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:46:54.549638 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:46:54.549660 (+ 22us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 03:46:54.550001 (+ 341us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 03:46:54.550013 (+ 12us) client_negotiation.cc:769] Sending connection context
0806 03:46:54.550071 (+ 58us) client_negotiation.cc:240] Negotiation successful
0806 03:46:54.550125 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":68}
I20260806 03:46:54.550182 15326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:46:54.547518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44506 (local address 127.11.253.194:45549)
0806 03:46:54.547583 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0806 03:46:54.547585 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:46:54.547595 (+ 10us) server_negotiation.cc:407] Connection header received
0806 03:46:54.547717 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:46:54.547719 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:46:54.547740 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:46:54.547830 (+ 90us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 03:46:54.547984 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:46:54.548477 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:46:54.549293 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:46:54.549488 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:46:54.549566 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:46:54.549765 (+ 199us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 03:46:54.549924 (+ 159us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 03:46:54.550023 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0806 03:46:54.550076 (+ 53us) server_negotiation.cc:299] Negotiation successful
0806 03:46:54.550099 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260806 03:46:54.552484 14837 raft_consensus.cc:1261] T fb0389a88b8a4fcb84a140efc6e5a09b P 2577e879d33e41ffb5a214cdae767802 [term 1 FOLLOWER]: Refusing update from remote peer 74eb6a2589da4205b5e5623842ef6faa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 03:46:54.552826 15236 raft_consensus.cc:1261] T fb0389a88b8a4fcb84a140efc6e5a09b P 18c08fd1a0d84a61ab7e6b98a82d3c11 [term 1 FOLLOWER]: Refusing update from remote peer 74eb6a2589da4205b5e5623842ef6faa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 03:46:54.553200 15347 consensus_queue.cc:1035] T fb0389a88b8a4fcb84a140efc6e5a09b P 74eb6a2589da4205b5e5623842ef6faa [LEADER]: Connected to new peer: Peer: permanent_uuid: "18c08fd1a0d84a61ab7e6b98a82d3c11" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 45903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 03:46:54.553457 15366 consensus_queue.cc:1035] T fb0389a88b8a4fcb84a140efc6e5a09b P 74eb6a2589da4205b5e5623842ef6faa [LEADER]: Connected to new peer: Peer: permanent_uuid: "2577e879d33e41ffb5a214cdae767802" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 41343 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 03:46:54.554750 15345 tablet_replica.cc:439] T fb0389a88b8a4fcb84a140efc6e5a09b P 18c08fd1a0d84a61ab7e6b98a82d3c11: TxnStatusTablet state changed. Reason: New leader 74eb6a2589da4205b5e5623842ef6faa. Latest consensus state: current_term: 1 leader_uuid: "74eb6a2589da4205b5e5623842ef6faa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2577e879d33e41ffb5a214cdae767802" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 41343 } } peers { permanent_uuid: "74eb6a2589da4205b5e5623842ef6faa" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 45549 } } peers { permanent_uuid: "18c08fd1a0d84a61ab7e6b98a82d3c11" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 45903 } } }
I20260806 03:46:54.554859 15345 tablet_replica.cc:442] T fb0389a88b8a4fcb84a140efc6e5a09b P 18c08fd1a0d84a61ab7e6b98a82d3c11: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 03:46:54.554840 15343 tablet_replica.cc:439] T fb0389a88b8a4fcb84a140efc6e5a09b P 2577e879d33e41ffb5a214cdae767802: TxnStatusTablet state changed. Reason: New leader 74eb6a2589da4205b5e5623842ef6faa. Latest consensus state: current_term: 1 leader_uuid: "74eb6a2589da4205b5e5623842ef6faa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2577e879d33e41ffb5a214cdae767802" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 41343 } } peers { permanent_uuid: "74eb6a2589da4205b5e5623842ef6faa" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 45549 } } peers { permanent_uuid: "18c08fd1a0d84a61ab7e6b98a82d3c11" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 45903 } } }
I20260806 03:46:54.554943 15343 tablet_replica.cc:442] T fb0389a88b8a4fcb84a140efc6e5a09b P 2577e879d33e41ffb5a214cdae767802: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 03:46:54.556176 15366 tablet_replica.cc:439] T fb0389a88b8a4fcb84a140efc6e5a09b P 74eb6a2589da4205b5e5623842ef6faa: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "74eb6a2589da4205b5e5623842ef6faa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2577e879d33e41ffb5a214cdae767802" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 41343 } } peers { permanent_uuid: "74eb6a2589da4205b5e5623842ef6faa" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 45549 } } peers { permanent_uuid: "18c08fd1a0d84a61ab7e6b98a82d3c11" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 45903 } } }
I20260806 03:46:54.556268 15366 tablet_replica.cc:442] T fb0389a88b8a4fcb84a140efc6e5a09b P 74eb6a2589da4205b5e5623842ef6faa: This TxnStatusTablet replica's current role is: LEADER
I20260806 03:46:54.556355 15345 tablet_replica.cc:439] T fb0389a88b8a4fcb84a140efc6e5a09b P 18c08fd1a0d84a61ab7e6b98a82d3c11: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "74eb6a2589da4205b5e5623842ef6faa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2577e879d33e41ffb5a214cdae767802" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 41343 } } peers { permanent_uuid: "74eb6a2589da4205b5e5623842ef6faa" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 45549 } } peers { permanent_uuid: "18c08fd1a0d84a61ab7e6b98a82d3c11" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 45903 } } }
I20260806 03:46:54.556381 15366 tablet_replica.cc:439] T fb0389a88b8a4fcb84a140efc6e5a09b P 74eb6a2589da4205b5e5623842ef6faa: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "74eb6a2589da4205b5e5623842ef6faa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2577e879d33e41ffb5a214cdae767802" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 41343 } } peers { permanent_uuid: "74eb6a2589da4205b5e5623842ef6faa" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 45549 } } peers { permanent_uuid: "18c08fd1a0d84a61ab7e6b98a82d3c11" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 45903 } } }
I20260806 03:46:54.556432 15345 tablet_replica.cc:442] T fb0389a88b8a4fcb84a140efc6e5a09b P 18c08fd1a0d84a61ab7e6b98a82d3c11: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 03:46:54.556447 15366 tablet_replica.cc:442] T fb0389a88b8a4fcb84a140efc6e5a09b P 74eb6a2589da4205b5e5623842ef6faa: This TxnStatusTablet replica's current role is: LEADER
I20260806 03:46:54.556814 15374 mvcc.cc:205] Tried to move back new op lower bound from 7315406907603959808 to 7315406907505332224. Current Snapshot: MvccSnapshot[applied={T|T < 7315406907603959808 or (T in {7315406907603959808})}]
I20260806 03:46:54.557430 15375 mvcc.cc:205] Tried to move back new op lower bound from 7315406907603959808 to 7315406907505332224. Current Snapshot: MvccSnapshot[applied={T|T < 7315406907603959808}]
I20260806 03:46:54.557552 15343 tablet_replica.cc:439] T fb0389a88b8a4fcb84a140efc6e5a09b P 2577e879d33e41ffb5a214cdae767802: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "74eb6a2589da4205b5e5623842ef6faa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2577e879d33e41ffb5a214cdae767802" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 41343 } } peers { permanent_uuid: "74eb6a2589da4205b5e5623842ef6faa" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 45549 } } peers { permanent_uuid: "18c08fd1a0d84a61ab7e6b98a82d3c11" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 45903 } } }
I20260806 03:46:54.557636 15343 tablet_replica.cc:442] T fb0389a88b8a4fcb84a140efc6e5a09b P 2577e879d33e41ffb5a214cdae767802: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 03:46:54.561435 15331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:46:54.558983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35390 (local address 127.11.253.195:45903)
0806 03:46:54.559029 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0806 03:46:54.559031 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:46:54.559040 (+ 9us) server_negotiation.cc:407] Connection header received
0806 03:46:54.559106 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:46:54.559107 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:46:54.559125 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:46:54.559202 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 03:46:54.559312 (+ 110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:46:54.559814 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:46:54.560598 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:46:54.560808 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:46:54.560887 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:46:54.561059 (+ 172us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 03:46:54.561170 (+ 111us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 03:46:54.561281 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0806 03:46:54.561328 (+ 47us) server_negotiation.cc:299] Negotiation successful
0806 03:46:54.561352 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260806 03:46:54.568511 15326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:46:54.565621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.253.195:38761 (local address 127.11.253.194:45549)
0806 03:46:54.565684 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0806 03:46:54.565686 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:46:54.565694 (+ 8us) server_negotiation.cc:407] Connection header received
0806 03:46:54.565714 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:46:54.565715 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:46:54.565724 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:46:54.565792 (+ 68us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 03:46:54.566060 (+ 268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:46:54.566581 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:46:54.567664 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:46:54.567860 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:46:54.567912 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:46:54.568210 (+ 298us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 03:46:54.568240 (+ 30us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 03:46:54.568347 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0806 03:46:54.568435 (+ 88us) server_negotiation.cc:299] Negotiation successful
0806 03:46:54.568455 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260806 03:46:54.568545 15307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:46:54.565361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.194:45549 (local address 127.11.253.195:38761)
0806 03:46:54.565444 (+ 83us) negotiation.cc:107] Waiting for socket to connect
0806 03:46:54.565457 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 03:46:54.565503 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:46:54.565821 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:46:54.565822 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:46:54.565824 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 03:46:54.565916 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:46:54.565918 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:46:54.566670 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:46:54.566671 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:46:54.567581 (+ 910us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:46:54.567583 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:46:54.567939 (+ 356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:46:54.567940 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:46:54.568011 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:46:54.568027 (+ 16us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 03:46:54.568369 (+ 342us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 03:46:54.568372 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 03:46:54.568421 (+ 49us) client_negotiation.cc:240] Negotiation successful
0806 03:46:54.568467 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42}
I20260806 03:46:54.571010 14837 raft_consensus.cc:1261] T 6ba15e72b6e240b88769688991489b1f P 2577e879d33e41ffb5a214cdae767802 [term 1 FOLLOWER]: Refusing update from remote peer 18c08fd1a0d84a61ab7e6b98a82d3c11: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 03:46:54.571101 15037 raft_consensus.cc:1261] T 6ba15e72b6e240b88769688991489b1f P 74eb6a2589da4205b5e5623842ef6faa [term 1 FOLLOWER]: Refusing update from remote peer 18c08fd1a0d84a61ab7e6b98a82d3c11: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 03:46:54.571316 15349 consensus_queue.cc:1035] T 6ba15e72b6e240b88769688991489b1f P 18c08fd1a0d84a61ab7e6b98a82d3c11 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2577e879d33e41ffb5a214cdae767802" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 41343 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 03:46:54.571455 15345 consensus_queue.cc:1035] T 6ba15e72b6e240b88769688991489b1f P 18c08fd1a0d84a61ab7e6b98a82d3c11 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74eb6a2589da4205b5e5623842ef6faa" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 45549 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 03:46:54.573681 15372 mvcc.cc:205] Tried to move back new op lower bound from 7315406907681030144 to 7315406906438623232. Current Snapshot: MvccSnapshot[applied={T|T < 7315406907681030144}]
I20260806 03:46:54.580561 15331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:46:54.577813 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.253.194:34673 (local address 127.11.253.195:45903)
0806 03:46:54.577898 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0806 03:46:54.577901 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 03:46:54.578040 (+ 139us) server_negotiation.cc:407] Connection header received
0806 03:46:54.578106 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:46:54.578106 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:46:54.578115 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:46:54.578160 (+ 45us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 03:46:54.578291 (+ 131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:46:54.578733 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:46:54.579903 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:46:54.580078 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:46:54.580125 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:46:54.580338 (+ 213us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 03:46:54.580366 (+ 28us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 03:46:54.580407 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0806 03:46:54.580490 (+ 83us) server_negotiation.cc:299] Negotiation successful
0806 03:46:54.580508 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260806 03:46:54.580623 15392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:46:54.577780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.195:45903 (local address 127.11.253.194:34673)
0806 03:46:54.577973 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0806 03:46:54.577985 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 03:46:54.578049 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:46:54.578165 (+ 116us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:46:54.578166 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:46:54.578167 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 03:46:54.578229 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:46:54.578231 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:46:54.578812 (+ 581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:46:54.578813 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:46:54.579812 (+ 999us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:46:54.579814 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:46:54.580149 (+ 335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:46:54.580150 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:46:54.580219 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:46:54.580235 (+ 16us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 03:46:54.580429 (+ 194us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 03:46:54.580431 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 03:46:54.580487 (+ 56us) client_negotiation.cc:240] Negotiation successful
0806 03:46:54.580530 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":122,"threads_started":1}
I20260806 03:46:54.590721 14605 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60630:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 03:46:54.590902 14605 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60630:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 03:46:54.591763 14605 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ae915b1dc3a24bdb8afeed530e88edf7: Sending DeleteTablet for 3 replicas of tablet 6ba15e72b6e240b88769688991489b1f
I20260806 03:46:54.592056 14801 tablet_service.cc:1514] Processing DeleteTablet for tablet 6ba15e72b6e240b88769688991489b1f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 03:46:54 UTC) from {username='kudu', principal='kudu/127.11.253.254@KRBTEST.COM'} at 127.0.0.1:55358
I20260806 03:46:54.592140 15002 tablet_service.cc:1514] Processing DeleteTablet for tablet 6ba15e72b6e240b88769688991489b1f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 03:46:54 UTC) from {username='kudu', principal='kudu/127.11.253.254@KRBTEST.COM'} at 127.0.0.1:44504
I20260806 03:46:54.592330 15204 tablet_service.cc:1514] Processing DeleteTablet for tablet 6ba15e72b6e240b88769688991489b1f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 03:46:54 UTC) from {username='kudu', principal='kudu/127.11.253.254@KRBTEST.COM'} at 127.0.0.1:35384
I20260806 03:46:54.592659 15397 tablet_replica.cc:330] T 6ba15e72b6e240b88769688991489b1f P 2577e879d33e41ffb5a214cdae767802: stopping tablet replica
I20260806 03:46:54.592636 14605 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60630:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260806 03:46:54.592787 15397 raft_consensus.cc:2229] T 6ba15e72b6e240b88769688991489b1f P 2577e879d33e41ffb5a214cdae767802 [term 1 FOLLOWER]: Raft consensus shutting down.
W20260806 03:46:54.592801 14605 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 03:46:54.592851 15397 raft_consensus.cc:2258] T 6ba15e72b6e240b88769688991489b1f P 2577e879d33e41ffb5a214cdae767802 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 03:46:54.593016 15399 tablet_replica.cc:330] T 6ba15e72b6e240b88769688991489b1f P 18c08fd1a0d84a61ab7e6b98a82d3c11: stopping tablet replica
I20260806 03:46:54.593119 15399 raft_consensus.cc:2229] T 6ba15e72b6e240b88769688991489b1f P 18c08fd1a0d84a61ab7e6b98a82d3c11 [term 1 LEADER]: Raft consensus shutting down.
I20260806 03:46:54.593215 15399 raft_consensus.cc:2258] T 6ba15e72b6e240b88769688991489b1f P 18c08fd1a0d84a61ab7e6b98a82d3c11 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 03:46:54.593743 15399 ts_tablet_manager.cc:1905] T 6ba15e72b6e240b88769688991489b1f P 18c08fd1a0d84a61ab7e6b98a82d3c11: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 03:46:54.594184 14801 tablet_service.cc:1467] Processing CreateTablet for tablet f5a6b11672c2478ea6b349c5a1da57c3 (DEFAULT_TABLE table=test-table [id=7f1e76d2c4414fd6b9bbd4e724c0e86b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 03:46:54.594180 15204 tablet_service.cc:1467] Processing CreateTablet for tablet f5a6b11672c2478ea6b349c5a1da57c3 (DEFAULT_TABLE table=test-table [id=7f1e76d2c4414fd6b9bbd4e724c0e86b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 03:46:54.594321 15204 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f5a6b11672c2478ea6b349c5a1da57c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:46:54.594321 14801 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f5a6b11672c2478ea6b349c5a1da57c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:46:54.595703 15397 ts_tablet_manager.cc:1905] T 6ba15e72b6e240b88769688991489b1f P 2577e879d33e41ffb5a214cdae767802: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 03:46:54.595799 15332 tablet_bootstrap.cc:492] T f5a6b11672c2478ea6b349c5a1da57c3 P 18c08fd1a0d84a61ab7e6b98a82d3c11: Bootstrap starting.
I20260806 03:46:54.595829 15333 tablet_bootstrap.cc:492] T f5a6b11672c2478ea6b349c5a1da57c3 P 2577e879d33e41ffb5a214cdae767802: Bootstrap starting.
I20260806 03:46:54.596300 15332 tablet_bootstrap.cc:654] T f5a6b11672c2478ea6b349c5a1da57c3 P 18c08fd1a0d84a61ab7e6b98a82d3c11: Neither blocks nor log segments found. Creating new log.
I20260806 03:46:54.596843 15002 tablet_service.cc:1467] Processing CreateTablet for tablet f5a6b11672c2478ea6b349c5a1da57c3 (DEFAULT_TABLE table=test-table [id=7f1e76d2c4414fd6b9bbd4e724c0e86b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 03:46:54.596935 15002 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f5a6b11672c2478ea6b349c5a1da57c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:46:54.597189 15397 ts_tablet_manager.cc:1918] T 6ba15e72b6e240b88769688991489b1f P 2577e879d33e41ffb5a214cdae767802: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 03:46:54.597252 15397 log.cc:1196] T 6ba15e72b6e240b88769688991489b1f P 2577e879d33e41ffb5a214cdae767802: Deleting WAL directory at /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785988004637995-12279-0/minicluster-data/ts-0/wal/wals/6ba15e72b6e240b88769688991489b1f
I20260806 03:46:54.597841 15341 tablet_bootstrap.cc:492] T f5a6b11672c2478ea6b349c5a1da57c3 P 74eb6a2589da4205b5e5623842ef6faa: Bootstrap starting.
I20260806 03:46:54.598145 15341 tablet_bootstrap.cc:654] T f5a6b11672c2478ea6b349c5a1da57c3 P 74eb6a2589da4205b5e5623842ef6faa: Neither blocks nor log segments found. Creating new log.
I20260806 03:46:54.598675 15333 tablet_bootstrap.cc:654] T f5a6b11672c2478ea6b349c5a1da57c3 P 2577e879d33e41ffb5a214cdae767802: Neither blocks nor log segments found. Creating new log.
I20260806 03:46:54.598779 15399 ts_tablet_manager.cc:1918] T 6ba15e72b6e240b88769688991489b1f P 18c08fd1a0d84a61ab7e6b98a82d3c11: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 03:46:54.598853 15399 log.cc:1196] T 6ba15e72b6e240b88769688991489b1f P 18c08fd1a0d84a61ab7e6b98a82d3c11: Deleting WAL directory at /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785988004637995-12279-0/minicluster-data/ts-2/wal/wals/6ba15e72b6e240b88769688991489b1f
I20260806 03:46:54.598991 15397 ts_tablet_manager.cc:1939] T 6ba15e72b6e240b88769688991489b1f P 2577e879d33e41ffb5a214cdae767802: Deleting consensus metadata
I20260806 03:46:54.599148 15399 ts_tablet_manager.cc:1939] T 6ba15e72b6e240b88769688991489b1f P 18c08fd1a0d84a61ab7e6b98a82d3c11: Deleting consensus metadata
I20260806 03:46:54.599401 14591 catalog_manager.cc:4841] TS 2577e879d33e41ffb5a214cdae767802 (127.11.253.193:41343): tablet 6ba15e72b6e240b88769688991489b1f (table test-table [id=9048e77224d74044a4f4698d5f4edd85]) successfully deleted
I20260806 03:46:54.599541 14588 catalog_manager.cc:4841] TS 18c08fd1a0d84a61ab7e6b98a82d3c11 (127.11.253.195:45903): tablet 6ba15e72b6e240b88769688991489b1f (table test-table [id=9048e77224d74044a4f4698d5f4edd85]) successfully deleted
I20260806 03:46:54.601992 15398 tablet_replica.cc:330] T 6ba15e72b6e240b88769688991489b1f P 74eb6a2589da4205b5e5623842ef6faa: stopping tablet replica
I20260806 03:46:54.602130 15398 raft_consensus.cc:2229] T 6ba15e72b6e240b88769688991489b1f P 74eb6a2589da4205b5e5623842ef6faa [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 03:46:54.602200 15398 raft_consensus.cc:2258] T 6ba15e72b6e240b88769688991489b1f P 74eb6a2589da4205b5e5623842ef6faa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 03:46:54.602317 15332 tablet_bootstrap.cc:492] T f5a6b11672c2478ea6b349c5a1da57c3 P 18c08fd1a0d84a61ab7e6b98a82d3c11: No bootstrap required, opened a new log
I20260806 03:46:54.602375 15332 ts_tablet_manager.cc:1397] T f5a6b11672c2478ea6b349c5a1da57c3 P 18c08fd1a0d84a61ab7e6b98a82d3c11: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.001s
I20260806 03:46:54.602533 15332 raft_consensus.cc:348] T f5a6b11672c2478ea6b349c5a1da57c3 P 18c08fd1a0d84a61ab7e6b98a82d3c11 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2577e879d33e41ffb5a214cdae767802" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 41343 } } peers { permanent_uuid: "74eb6a2589da4205b5e5623842ef6faa" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 45549 } } peers { permanent_uuid: "18c08fd1a0d84a61ab7e6b98a82d3c11" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 45903 } }
I20260806 03:46:54.602612 15332 raft_consensus.cc:374] T f5a6b11672c2478ea6b349c5a1da57c3 P 18c08fd1a0d84a61ab7e6b98a82d3c11 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:46:54.602638 15332 raft_consensus.cc:729] T f5a6b11672c2478ea6b349c5a1da57c3 P 18c08fd1a0d84a61ab7e6b98a82d3c11 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 18c08fd1a0d84a61ab7e6b98a82d3c11, State: Initialized, Role: FOLLOWER
I20260806 03:46:54.602644 15398 ts_tablet_manager.cc:1905] T 6ba15e72b6e240b88769688991489b1f P 74eb6a2589da4205b5e5623842ef6faa: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 03:46:54.602680 15332 consensus_queue.cc:260] T f5a6b11672c2478ea6b349c5a1da57c3 P 18c08fd1a0d84a61ab7e6b98a82d3c11 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2577e879d33e41ffb5a214cdae767802" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 41343 } } peers { permanent_uuid: "74eb6a2589da4205b5e5623842ef6faa" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 45549 } } peers { permanent_uuid: "18c08fd1a0d84a61ab7e6b98a82d3c11" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 45903 } }
I20260806 03:46:54.602785 15332 ts_tablet_manager.cc:1428] T f5a6b11672c2478ea6b349c5a1da57c3 P 18c08fd1a0d84a61ab7e6b98a82d3c11: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 03:46:54.603529 15333 tablet_bootstrap.cc:492] T f5a6b11672c2478ea6b349c5a1da57c3 P 2577e879d33e41ffb5a214cdae767802: No bootstrap required, opened a new log
I20260806 03:46:54.603608 15333 ts_tablet_manager.cc:1397] T f5a6b11672c2478ea6b349c5a1da57c3 P 2577e879d33e41ffb5a214cdae767802: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.003s
I20260806 03:46:54.603686 15341 tablet_bootstrap.cc:492] T f5a6b11672c2478ea6b349c5a1da57c3 P 74eb6a2589da4205b5e5623842ef6faa: No bootstrap required, opened a new log
I20260806 03:46:54.603725 15341 ts_tablet_manager.cc:1397] T f5a6b11672c2478ea6b349c5a1da57c3 P 74eb6a2589da4205b5e5623842ef6faa: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.000s
I20260806 03:46:54.603760 15333 raft_consensus.cc:348] T f5a6b11672c2478ea6b349c5a1da57c3 P 2577e879d33e41ffb5a214cdae767802 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2577e879d33e41ffb5a214cdae767802" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 41343 } } peers { permanent_uuid: "74eb6a2589da4205b5e5623842ef6faa" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 45549 } } peers { permanent_uuid: "18c08fd1a0d84a61ab7e6b98a82d3c11" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 45903 } }
I20260806 03:46:54.603848 15333 raft_consensus.cc:374] T f5a6b11672c2478ea6b349c5a1da57c3 P 2577e879d33e41ffb5a214cdae767802 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:46:54.603868 15333 raft_consensus.cc:729] T f5a6b11672c2478ea6b349c5a1da57c3 P 2577e879d33e41ffb5a214cdae767802 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2577e879d33e41ffb5a214cdae767802, State: Initialized, Role: FOLLOWER
I20260806 03:46:54.603892 15341 raft_consensus.cc:348] T f5a6b11672c2478ea6b349c5a1da57c3 P 74eb6a2589da4205b5e5623842ef6faa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2577e879d33e41ffb5a214cdae767802" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 41343 } } peers { permanent_uuid: "74eb6a2589da4205b5e5623842ef6faa" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 45549 } } peers { permanent_uuid: "18c08fd1a0d84a61ab7e6b98a82d3c11" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 45903 } }
I20260806 03:46:54.603917 15333 consensus_queue.cc:260] T f5a6b11672c2478ea6b349c5a1da57c3 P 2577e879d33e41ffb5a214cdae767802 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2577e879d33e41ffb5a214cdae767802" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 41343 } } peers { permanent_uuid: "74eb6a2589da4205b5e5623842ef6faa" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 45549 } } peers { permanent_uuid: "18c08fd1a0d84a61ab7e6b98a82d3c11" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 45903 } }
I20260806 03:46:54.603964 15341 raft_consensus.cc:374] T f5a6b11672c2478ea6b349c5a1da57c3 P 74eb6a2589da4205b5e5623842ef6faa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:46:54.603992 15341 raft_consensus.cc:729] T f5a6b11672c2478ea6b349c5a1da57c3 P 74eb6a2589da4205b5e5623842ef6faa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74eb6a2589da4205b5e5623842ef6faa, State: Initialized, Role: FOLLOWER
I20260806 03:46:54.604012 15333 ts_tablet_manager.cc:1428] T f5a6b11672c2478ea6b349c5a1da57c3 P 2577e879d33e41ffb5a214cdae767802: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 03:46:54.604032 15341 consensus_queue.cc:260] T f5a6b11672c2478ea6b349c5a1da57c3 P 74eb6a2589da4205b5e5623842ef6faa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2577e879d33e41ffb5a214cdae767802" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 41343 } } peers { permanent_uuid: "74eb6a2589da4205b5e5623842ef6faa" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 45549 } } peers { permanent_uuid: "18c08fd1a0d84a61ab7e6b98a82d3c11" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 45903 } }
I20260806 03:46:54.604123 15341 ts_tablet_manager.cc:1428] T f5a6b11672c2478ea6b349c5a1da57c3 P 74eb6a2589da4205b5e5623842ef6faa: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260806 03:46:54.604830 15398 ts_tablet_manager.cc:1918] T 6ba15e72b6e240b88769688991489b1f P 74eb6a2589da4205b5e5623842ef6faa: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 03:46:54.604907 15398 log.cc:1196] T 6ba15e72b6e240b88769688991489b1f P 74eb6a2589da4205b5e5623842ef6faa: Deleting WAL directory at /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785988004637995-12279-0/minicluster-data/ts-1/wal/wals/6ba15e72b6e240b88769688991489b1f
I20260806 03:46:54.605223 15398 ts_tablet_manager.cc:1939] T 6ba15e72b6e240b88769688991489b1f P 74eb6a2589da4205b5e5623842ef6faa: Deleting consensus metadata
I20260806 03:46:54.605705 14590 catalog_manager.cc:4841] TS 74eb6a2589da4205b5e5623842ef6faa (127.11.253.194:45549): tablet 6ba15e72b6e240b88769688991489b1f (table test-table [id=9048e77224d74044a4f4698d5f4edd85]) successfully deleted
I20260806 03:46:54.864910 15347 raft_consensus.cc:482] T f5a6b11672c2478ea6b349c5a1da57c3 P 74eb6a2589da4205b5e5623842ef6faa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 03:46:54.865015 15347 raft_consensus.cc:504] T f5a6b11672c2478ea6b349c5a1da57c3 P 74eb6a2589da4205b5e5623842ef6faa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2577e879d33e41ffb5a214cdae767802" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 41343 } } peers { permanent_uuid: "74eb6a2589da4205b5e5623842ef6faa" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 45549 } } peers { permanent_uuid: "18c08fd1a0d84a61ab7e6b98a82d3c11" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 45903 } }
I20260806 03:46:54.865196 15347 leader_election.cc:288] T f5a6b11672c2478ea6b349c5a1da57c3 P 74eb6a2589da4205b5e5623842ef6faa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2577e879d33e41ffb5a214cdae767802 (127.11.253.193:41343), 18c08fd1a0d84a61ab7e6b98a82d3c11 (127.11.253.195:45903)
I20260806 03:46:54.865453 15236 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f5a6b11672c2478ea6b349c5a1da57c3" candidate_uuid: "74eb6a2589da4205b5e5623842ef6faa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "18c08fd1a0d84a61ab7e6b98a82d3c11" is_pre_election: true
I20260806 03:46:54.865554 15236 raft_consensus.cc:2454] T f5a6b11672c2478ea6b349c5a1da57c3 P 18c08fd1a0d84a61ab7e6b98a82d3c11 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74eb6a2589da4205b5e5623842ef6faa in term 0.
I20260806 03:46:54.865559 14837 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f5a6b11672c2478ea6b349c5a1da57c3" candidate_uuid: "74eb6a2589da4205b5e5623842ef6faa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2577e879d33e41ffb5a214cdae767802" is_pre_election: true
I20260806 03:46:54.865710 14837 raft_consensus.cc:2454] T f5a6b11672c2478ea6b349c5a1da57c3 P 2577e879d33e41ffb5a214cdae767802 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74eb6a2589da4205b5e5623842ef6faa in term 0.
I20260806 03:46:54.865762 14935 leader_election.cc:302] T f5a6b11672c2478ea6b349c5a1da57c3 P 74eb6a2589da4205b5e5623842ef6faa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 18c08fd1a0d84a61ab7e6b98a82d3c11, 74eb6a2589da4205b5e5623842ef6faa; no voters:
I20260806 03:46:54.865947 15347 raft_consensus.cc:2781] T f5a6b11672c2478ea6b349c5a1da57c3 P 74eb6a2589da4205b5e5623842ef6faa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 03:46:54.865991 15347 raft_consensus.cc:482] T f5a6b11672c2478ea6b349c5a1da57c3 P 74eb6a2589da4205b5e5623842ef6faa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 03:46:54.866025 15347 raft_consensus.cc:3037] T f5a6b11672c2478ea6b349c5a1da57c3 P 74eb6a2589da4205b5e5623842ef6faa [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:46:54.866720 15347 raft_consensus.cc:504] T f5a6b11672c2478ea6b349c5a1da57c3 P 74eb6a2589da4205b5e5623842ef6faa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2577e879d33e41ffb5a214cdae767802" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 41343 } } peers { permanent_uuid: "74eb6a2589da4205b5e5623842ef6faa" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 45549 } } peers { permanent_uuid: "18c08fd1a0d84a61ab7e6b98a82d3c11" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 45903 } }
I20260806 03:46:54.866881 15347 leader_election.cc:288] T f5a6b11672c2478ea6b349c5a1da57c3 P 74eb6a2589da4205b5e5623842ef6faa [CANDIDATE]: Term 1 election: Requested vote from peers 2577e879d33e41ffb5a214cdae767802 (127.11.253.193:41343), 18c08fd1a0d84a61ab7e6b98a82d3c11 (127.11.253.195:45903)
I20260806 03:46:54.867045 15236 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f5a6b11672c2478ea6b349c5a1da57c3" candidate_uuid: "74eb6a2589da4205b5e5623842ef6faa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "18c08fd1a0d84a61ab7e6b98a82d3c11"
I20260806 03:46:54.867151 15236 raft_consensus.cc:3037] T f5a6b11672c2478ea6b349c5a1da57c3 P 18c08fd1a0d84a61ab7e6b98a82d3c11 [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:46:54.867113 14837 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f5a6b11672c2478ea6b349c5a1da57c3" candidate_uuid: "74eb6a2589da4205b5e5623842ef6faa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2577e879d33e41ffb5a214cdae767802"
I20260806 03:46:54.867307 14837 raft_consensus.cc:3037] T f5a6b11672c2478ea6b349c5a1da57c3 P 2577e879d33e41ffb5a214cdae767802 [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:46:54.867841 15236 raft_consensus.cc:2454] T f5a6b11672c2478ea6b349c5a1da57c3 P 18c08fd1a0d84a61ab7e6b98a82d3c11 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74eb6a2589da4205b5e5623842ef6faa in term 1.
I20260806 03:46:54.868054 14935 leader_election.cc:302] T f5a6b11672c2478ea6b349c5a1da57c3 P 74eb6a2589da4205b5e5623842ef6faa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 18c08fd1a0d84a61ab7e6b98a82d3c11, 74eb6a2589da4205b5e5623842ef6faa; no voters:
I20260806 03:46:54.868078 14837 raft_consensus.cc:2454] T f5a6b11672c2478ea6b349c5a1da57c3 P 2577e879d33e41ffb5a214cdae767802 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74eb6a2589da4205b5e5623842ef6faa in term 1.
I20260806 03:46:54.868212 15347 raft_consensus.cc:2781] T f5a6b11672c2478ea6b349c5a1da57c3 P 74eb6a2589da4205b5e5623842ef6faa [term 1 FOLLOWER]: Leader election won for term 1
I20260806 03:46:54.868283 15347 raft_consensus.cc:686] T f5a6b11672c2478ea6b349c5a1da57c3 P 74eb6a2589da4205b5e5623842ef6faa [term 1 LEADER]: Becoming Leader. State: Replica: 74eb6a2589da4205b5e5623842ef6faa, State: Running, Role: LEADER
I20260806 03:46:54.868350 15347 consensus_queue.cc:237] T f5a6b11672c2478ea6b349c5a1da57c3 P 74eb6a2589da4205b5e5623842ef6faa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2577e879d33e41ffb5a214cdae767802" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 41343 } } peers { permanent_uuid: "74eb6a2589da4205b5e5623842ef6faa" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 45549 } } peers { permanent_uuid: "18c08fd1a0d84a61ab7e6b98a82d3c11" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 45903 } }
I20260806 03:46:54.868999 14605 catalog_manager.cc:5495] T f5a6b11672c2478ea6b349c5a1da57c3 P 74eb6a2589da4205b5e5623842ef6faa reported cstate change: term changed from 0 to 1, leader changed from <none> to 74eb6a2589da4205b5e5623842ef6faa (127.11.253.194). New cstate: current_term: 1 leader_uuid: "74eb6a2589da4205b5e5623842ef6faa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2577e879d33e41ffb5a214cdae767802" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 41343 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "74eb6a2589da4205b5e5623842ef6faa" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 45549 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "18c08fd1a0d84a61ab7e6b98a82d3c11" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 45903 } health_report { overall_health: UNKNOWN } } }
I20260806 03:46:54.888985 15326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:46:54.886675 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44520 (local address 127.11.253.194:45549)
0806 03:46:54.886725 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0806 03:46:54.886727 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:46:54.886736 (+ 9us) server_negotiation.cc:407] Connection header received
0806 03:46:54.886761 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:46:54.886770 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:46:54.886784 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:46:54.886857 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 03:46:54.886987 (+ 130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:46:54.887458 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:46:54.888261 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:46:54.888462 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:46:54.888538 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:46:54.888689 (+ 151us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 03:46:54.888728 (+ 39us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 03:46:54.888798 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0806 03:46:54.888865 (+ 67us) server_negotiation.cc:299] Negotiation successful
0806 03:46:54.888889 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260806 03:46:54.893240 15326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:46:54.891241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.253.194:49481 (local address 127.11.253.194:45549)
0806 03:46:54.891332 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0806 03:46:54.891334 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:46:54.891342 (+ 8us) server_negotiation.cc:407] Connection header received
0806 03:46:54.891361 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:46:54.891362 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:46:54.891371 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:46:54.891425 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 03:46:54.891599 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:46:54.892027 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:46:54.892717 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:46:54.892897 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:46:54.892945 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:46:54.893058 (+ 113us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 03:46:54.893079 (+ 21us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 03:46:54.893111 (+ 32us) server_negotiation.cc:1035] Waiting for connection context
0806 03:46:54.893163 (+ 52us) server_negotiation.cc:299] Negotiation successful
0806 03:46:54.893175 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260806 03:46:54.893350 15392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:46:54.891186 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.194:45549 (local address 127.11.253.194:49481)
0806 03:46:54.891253 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0806 03:46:54.891270 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 03:46:54.891306 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:46:54.891489 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:46:54.891490 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:46:54.891491 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 03:46:54.891530 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:46:54.891531 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:46:54.892111 (+ 580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:46:54.892112 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:46:54.892647 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:46:54.892648 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:46:54.892963 (+ 315us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:46:54.892964 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:46:54.893003 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:46:54.893008 (+ 5us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 03:46:54.893127 (+ 119us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 03:46:54.893128 (+ 1us) client_negotiation.cc:769] Sending connection context
0806 03:46:54.893163 (+ 35us) client_negotiation.cc:240] Negotiation successful
0806 03:46:54.893202 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44}
I20260806 03:46:54.895572 15236 raft_consensus.cc:1261] T f5a6b11672c2478ea6b349c5a1da57c3 P 18c08fd1a0d84a61ab7e6b98a82d3c11 [term 1 FOLLOWER]: Refusing update from remote peer 74eb6a2589da4205b5e5623842ef6faa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 03:46:54.895586 14837 raft_consensus.cc:1261] T f5a6b11672c2478ea6b349c5a1da57c3 P 2577e879d33e41ffb5a214cdae767802 [term 1 FOLLOWER]: Refusing update from remote peer 74eb6a2589da4205b5e5623842ef6faa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 03:46:54.895781 15405 consensus_queue.cc:1035] T f5a6b11672c2478ea6b349c5a1da57c3 P 74eb6a2589da4205b5e5623842ef6faa [LEADER]: Connected to new peer: Peer: permanent_uuid: "18c08fd1a0d84a61ab7e6b98a82d3c11" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 45903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 03:46:54.895890 15347 consensus_queue.cc:1035] T f5a6b11672c2478ea6b349c5a1da57c3 P 74eb6a2589da4205b5e5623842ef6faa [LEADER]: Connected to new peer: Peer: permanent_uuid: "2577e879d33e41ffb5a214cdae767802" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 41343 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260806 03:46:54.900537 15372 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260806 03:46:56.419754 14606 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60630:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 03:46:56.419857 14606 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60630:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 03:46:56.420709 14606 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ae915b1dc3a24bdb8afeed530e88edf7: Sending DeleteTablet for 3 replicas of tablet f5a6b11672c2478ea6b349c5a1da57c3
I20260806 03:46:56.420989 14801 tablet_service.cc:1514] Processing DeleteTablet for tablet f5a6b11672c2478ea6b349c5a1da57c3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 03:46:56 UTC) from {username='kudu', principal='kudu/127.11.253.254@KRBTEST.COM'} at 127.0.0.1:55358
I20260806 03:46:56.421005 15002 tablet_service.cc:1514] Processing DeleteTablet for tablet f5a6b11672c2478ea6b349c5a1da57c3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 03:46:56 UTC) from {username='kudu', principal='kudu/127.11.253.254@KRBTEST.COM'} at 127.0.0.1:44504
I20260806 03:46:56.421012 15204 tablet_service.cc:1514] Processing DeleteTablet for tablet f5a6b11672c2478ea6b349c5a1da57c3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 03:46:56 UTC) from {username='kudu', principal='kudu/127.11.253.254@KRBTEST.COM'} at 127.0.0.1:35384
I20260806 03:46:56.421247 15442 tablet_replica.cc:330] T f5a6b11672c2478ea6b349c5a1da57c3 P 74eb6a2589da4205b5e5623842ef6faa: stopping tablet replica
I20260806 03:46:56.421363 15442 raft_consensus.cc:2229] T f5a6b11672c2478ea6b349c5a1da57c3 P 74eb6a2589da4205b5e5623842ef6faa [term 1 LEADER]: Raft consensus shutting down.
I20260806 03:46:56.421450 15442 raft_consensus.cc:2258] T f5a6b11672c2478ea6b349c5a1da57c3 P 74eb6a2589da4205b5e5623842ef6faa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 03:46:56.421566 15446 tablet_replica.cc:330] T f5a6b11672c2478ea6b349c5a1da57c3 P 18c08fd1a0d84a61ab7e6b98a82d3c11: stopping tablet replica
I20260806 03:46:56.421617 15446 raft_consensus.cc:2229] T f5a6b11672c2478ea6b349c5a1da57c3 P 18c08fd1a0d84a61ab7e6b98a82d3c11 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 03:46:56.421648 15446 raft_consensus.cc:2258] T f5a6b11672c2478ea6b349c5a1da57c3 P 18c08fd1a0d84a61ab7e6b98a82d3c11 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 03:46:56.421746 15444 tablet_replica.cc:330] T f5a6b11672c2478ea6b349c5a1da57c3 P 2577e879d33e41ffb5a214cdae767802: stopping tablet replica
I20260806 03:46:56.421800 15444 raft_consensus.cc:2229] T f5a6b11672c2478ea6b349c5a1da57c3 P 2577e879d33e41ffb5a214cdae767802 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 03:46:56.421854 15444 raft_consensus.cc:2258] T f5a6b11672c2478ea6b349c5a1da57c3 P 2577e879d33e41ffb5a214cdae767802 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 03:46:56.423309 15442 ts_tablet_manager.cc:1905] T f5a6b11672c2478ea6b349c5a1da57c3 P 74eb6a2589da4205b5e5623842ef6faa: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 03:46:56.423463 15444 ts_tablet_manager.cc:1905] T f5a6b11672c2478ea6b349c5a1da57c3 P 2577e879d33e41ffb5a214cdae767802: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 03:46:56.424698 15446 ts_tablet_manager.cc:1905] T f5a6b11672c2478ea6b349c5a1da57c3 P 18c08fd1a0d84a61ab7e6b98a82d3c11: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 03:46:56.425416 15442 ts_tablet_manager.cc:1918] T f5a6b11672c2478ea6b349c5a1da57c3 P 74eb6a2589da4205b5e5623842ef6faa: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 03:46:56.425431 15444 ts_tablet_manager.cc:1918] T f5a6b11672c2478ea6b349c5a1da57c3 P 2577e879d33e41ffb5a214cdae767802: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 03:46:56.425482 15442 log.cc:1196] T f5a6b11672c2478ea6b349c5a1da57c3 P 74eb6a2589da4205b5e5623842ef6faa: Deleting WAL directory at /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785988004637995-12279-0/minicluster-data/ts-1/wal/wals/f5a6b11672c2478ea6b349c5a1da57c3
I20260806 03:46:56.425498 15444 log.cc:1196] T f5a6b11672c2478ea6b349c5a1da57c3 P 2577e879d33e41ffb5a214cdae767802: Deleting WAL directory at /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785988004637995-12279-0/minicluster-data/ts-0/wal/wals/f5a6b11672c2478ea6b349c5a1da57c3
I20260806 03:46:56.425757 15442 ts_tablet_manager.cc:1939] T f5a6b11672c2478ea6b349c5a1da57c3 P 74eb6a2589da4205b5e5623842ef6faa: Deleting consensus metadata
I20260806 03:46:56.425791 15444 ts_tablet_manager.cc:1939] T f5a6b11672c2478ea6b349c5a1da57c3 P 2577e879d33e41ffb5a214cdae767802: Deleting consensus metadata
I20260806 03:46:56.426172 14590 catalog_manager.cc:4841] TS 74eb6a2589da4205b5e5623842ef6faa (127.11.253.194:45549): tablet f5a6b11672c2478ea6b349c5a1da57c3 (table test-table [id=7f1e76d2c4414fd6b9bbd4e724c0e86b]) successfully deleted
I20260806 03:46:56.426173 14591 catalog_manager.cc:4841] TS 2577e879d33e41ffb5a214cdae767802 (127.11.253.193:41343): tablet f5a6b11672c2478ea6b349c5a1da57c3 (table test-table [id=7f1e76d2c4414fd6b9bbd4e724c0e86b]) successfully deleted
I20260806 03:46:56.427306 15446 ts_tablet_manager.cc:1918] T f5a6b11672c2478ea6b349c5a1da57c3 P 18c08fd1a0d84a61ab7e6b98a82d3c11: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 03:46:56.427383 15446 log.cc:1196] T f5a6b11672c2478ea6b349c5a1da57c3 P 18c08fd1a0d84a61ab7e6b98a82d3c11: Deleting WAL directory at /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785988004637995-12279-0/minicluster-data/ts-2/wal/wals/f5a6b11672c2478ea6b349c5a1da57c3
I20260806 03:46:56.427590 15446 ts_tablet_manager.cc:1939] T f5a6b11672c2478ea6b349c5a1da57c3 P 18c08fd1a0d84a61ab7e6b98a82d3c11: Deleting consensus metadata
I20260806 03:46:56.427917 14588 catalog_manager.cc:4841] TS 18c08fd1a0d84a61ab7e6b98a82d3c11 (127.11.253.195:45903): tablet f5a6b11672c2478ea6b349c5a1da57c3 (table test-table [id=7f1e76d2c4414fd6b9bbd4e724c0e86b]) successfully deleted
Aug 06 03:46:56 dist-test-slave-6ntk krb5kdc[14550](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988016, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 06 03:46:56 dist-test-slave-6ntk krb5kdc[14550](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988016, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.11.253.254@KRBTEST.COM
I20260806 03:46:56.452996 15461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:46:56.442238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60644 (local address 127.11.253.254:42173)
0806 03:46:56.442409 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0806 03:46:56.442411 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:46:56.442454 (+ 43us) server_negotiation.cc:407] Connection header received
0806 03:46:56.442555 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:46:56.442555 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:46:56.442567 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:46:56.442629 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:46:56.443254 (+ 625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:46:56.443780 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:46:56.444677 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:46:56.444885 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:46:56.444935 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:46:56.450402 (+ 5467us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:46:56.450403 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:46:56.450404 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:46:56.450412 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:46:56.451820 (+ 1408us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:46:56.452209 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:46:56.452210 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:46:56.452210 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:46:56.452267 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:46:56.452477 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:46:56.452477 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:46:56.452478 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:46:56.452583 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:46:56.452660 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0806 03:46:56.452778 (+ 118us) server_negotiation.cc:299] Negotiation successful
0806 03:46:56.452858 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":120,"threads_started":1}
I20260806 03:46:56.454624 14606 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60644:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 03:46:56.454773 14606 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 03:46:56.456297 15204 tablet_service.cc:1467] Processing CreateTablet for tablet 9ae270f9e8da4c02bf09c754a0d33304 (DEFAULT_TABLE table=test-table [id=4461443b4b5b45e7a70c2bd95dd48f67]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 03:46:56.456300 15002 tablet_service.cc:1467] Processing CreateTablet for tablet 9ae270f9e8da4c02bf09c754a0d33304 (DEFAULT_TABLE table=test-table [id=4461443b4b5b45e7a70c2bd95dd48f67]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 03:46:56.456322 14801 tablet_service.cc:1467] Processing CreateTablet for tablet 9ae270f9e8da4c02bf09c754a0d33304 (DEFAULT_TABLE table=test-table [id=4461443b4b5b45e7a70c2bd95dd48f67]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 03:46:56.456452 14801 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9ae270f9e8da4c02bf09c754a0d33304. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:46:56.456455 15204 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9ae270f9e8da4c02bf09c754a0d33304. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:46:56.456465 15002 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9ae270f9e8da4c02bf09c754a0d33304. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:46:56.457636 15462 tablet_bootstrap.cc:492] T 9ae270f9e8da4c02bf09c754a0d33304 P 74eb6a2589da4205b5e5623842ef6faa: Bootstrap starting.
I20260806 03:46:56.457983 15466 tablet_bootstrap.cc:492] T 9ae270f9e8da4c02bf09c754a0d33304 P 2577e879d33e41ffb5a214cdae767802: Bootstrap starting.
I20260806 03:46:56.458177 15462 tablet_bootstrap.cc:654] T 9ae270f9e8da4c02bf09c754a0d33304 P 74eb6a2589da4205b5e5623842ef6faa: Neither blocks nor log segments found. Creating new log.
I20260806 03:46:56.458453 15466 tablet_bootstrap.cc:654] T 9ae270f9e8da4c02bf09c754a0d33304 P 2577e879d33e41ffb5a214cdae767802: Neither blocks nor log segments found. Creating new log.
I20260806 03:46:56.458839 15465 tablet_bootstrap.cc:492] T 9ae270f9e8da4c02bf09c754a0d33304 P 18c08fd1a0d84a61ab7e6b98a82d3c11: Bootstrap starting.
I20260806 03:46:56.458902 15462 tablet_bootstrap.cc:492] T 9ae270f9e8da4c02bf09c754a0d33304 P 74eb6a2589da4205b5e5623842ef6faa: No bootstrap required, opened a new log
I20260806 03:46:56.458943 15462 ts_tablet_manager.cc:1397] T 9ae270f9e8da4c02bf09c754a0d33304 P 74eb6a2589da4205b5e5623842ef6faa: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260806 03:46:56.459116 15466 tablet_bootstrap.cc:492] T 9ae270f9e8da4c02bf09c754a0d33304 P 2577e879d33e41ffb5a214cdae767802: No bootstrap required, opened a new log
I20260806 03:46:56.459112 15462 raft_consensus.cc:348] T 9ae270f9e8da4c02bf09c754a0d33304 P 74eb6a2589da4205b5e5623842ef6faa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2577e879d33e41ffb5a214cdae767802" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 41343 } } peers { permanent_uuid: "74eb6a2589da4205b5e5623842ef6faa" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 45549 } } peers { permanent_uuid: "18c08fd1a0d84a61ab7e6b98a82d3c11" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 45903 } }
I20260806 03:46:56.459182 15466 ts_tablet_manager.cc:1397] T 9ae270f9e8da4c02bf09c754a0d33304 P 2577e879d33e41ffb5a214cdae767802: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260806 03:46:56.459203 15462 raft_consensus.cc:374] T 9ae270f9e8da4c02bf09c754a0d33304 P 74eb6a2589da4205b5e5623842ef6faa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:46:56.459229 15462 raft_consensus.cc:729] T 9ae270f9e8da4c02bf09c754a0d33304 P 74eb6a2589da4205b5e5623842ef6faa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74eb6a2589da4205b5e5623842ef6faa, State: Initialized, Role: FOLLOWER
I20260806 03:46:56.459242 15465 tablet_bootstrap.cc:654] T 9ae270f9e8da4c02bf09c754a0d33304 P 18c08fd1a0d84a61ab7e6b98a82d3c11: Neither blocks nor log segments found. Creating new log.
I20260806 03:46:56.459309 15462 consensus_queue.cc:260] T 9ae270f9e8da4c02bf09c754a0d33304 P 74eb6a2589da4205b5e5623842ef6faa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2577e879d33e41ffb5a214cdae767802" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 41343 } } peers { permanent_uuid: "74eb6a2589da4205b5e5623842ef6faa" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 45549 } } peers { permanent_uuid: "18c08fd1a0d84a61ab7e6b98a82d3c11" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 45903 } }
I20260806 03:46:56.459529 15462 ts_tablet_manager.cc:1428] T 9ae270f9e8da4c02bf09c754a0d33304 P 74eb6a2589da4205b5e5623842ef6faa: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 03:46:56.459553 15466 raft_consensus.cc:348] T 9ae270f9e8da4c02bf09c754a0d33304 P 2577e879d33e41ffb5a214cdae767802 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2577e879d33e41ffb5a214cdae767802" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 41343 } } peers { permanent_uuid: "74eb6a2589da4205b5e5623842ef6faa" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 45549 } } peers { permanent_uuid: "18c08fd1a0d84a61ab7e6b98a82d3c11" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 45903 } }
I20260806 03:46:56.459635 15466 raft_consensus.cc:374] T 9ae270f9e8da4c02bf09c754a0d33304 P 2577e879d33e41ffb5a214cdae767802 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:46:56.459661 15466 raft_consensus.cc:729] T 9ae270f9e8da4c02bf09c754a0d33304 P 2577e879d33e41ffb5a214cdae767802 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2577e879d33e41ffb5a214cdae767802, State: Initialized, Role: FOLLOWER
I20260806 03:46:56.459702 15466 consensus_queue.cc:260] T 9ae270f9e8da4c02bf09c754a0d33304 P 2577e879d33e41ffb5a214cdae767802 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2577e879d33e41ffb5a214cdae767802" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 41343 } } peers { permanent_uuid: "74eb6a2589da4205b5e5623842ef6faa" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 45549 } } peers { permanent_uuid: "18c08fd1a0d84a61ab7e6b98a82d3c11" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 45903 } }
I20260806 03:46:56.459897 15466 ts_tablet_manager.cc:1428] T 9ae270f9e8da4c02bf09c754a0d33304 P 2577e879d33e41ffb5a214cdae767802: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 03:46:56.460088 15465 tablet_bootstrap.cc:492] T 9ae270f9e8da4c02bf09c754a0d33304 P 18c08fd1a0d84a61ab7e6b98a82d3c11: No bootstrap required, opened a new log
I20260806 03:46:56.460143 15465 ts_tablet_manager.cc:1397] T 9ae270f9e8da4c02bf09c754a0d33304 P 18c08fd1a0d84a61ab7e6b98a82d3c11: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260806 03:46:56.460259 15465 raft_consensus.cc:348] T 9ae270f9e8da4c02bf09c754a0d33304 P 18c08fd1a0d84a61ab7e6b98a82d3c11 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2577e879d33e41ffb5a214cdae767802" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 41343 } } peers { permanent_uuid: "74eb6a2589da4205b5e5623842ef6faa" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 45549 } } peers { permanent_uuid: "18c08fd1a0d84a61ab7e6b98a82d3c11" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 45903 } }
I20260806 03:46:56.460340 15465 raft_consensus.cc:374] T 9ae270f9e8da4c02bf09c754a0d33304 P 18c08fd1a0d84a61ab7e6b98a82d3c11 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:46:56.460364 15465 raft_consensus.cc:729] T 9ae270f9e8da4c02bf09c754a0d33304 P 18c08fd1a0d84a61ab7e6b98a82d3c11 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 18c08fd1a0d84a61ab7e6b98a82d3c11, State: Initialized, Role: FOLLOWER
I20260806 03:46:56.460414 15465 consensus_queue.cc:260] T 9ae270f9e8da4c02bf09c754a0d33304 P 18c08fd1a0d84a61ab7e6b98a82d3c11 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2577e879d33e41ffb5a214cdae767802" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 41343 } } peers { permanent_uuid: "74eb6a2589da4205b5e5623842ef6faa" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 45549 } } peers { permanent_uuid: "18c08fd1a0d84a61ab7e6b98a82d3c11" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 45903 } }
I20260806 03:46:56.460614 15465 ts_tablet_manager.cc:1428] T 9ae270f9e8da4c02bf09c754a0d33304 P 18c08fd1a0d84a61ab7e6b98a82d3c11: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260806 03:46:56.486726 15424 raft_consensus.cc:482] T 9ae270f9e8da4c02bf09c754a0d33304 P 74eb6a2589da4205b5e5623842ef6faa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 03:46:56.486852 15424 raft_consensus.cc:504] T 9ae270f9e8da4c02bf09c754a0d33304 P 74eb6a2589da4205b5e5623842ef6faa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2577e879d33e41ffb5a214cdae767802" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 41343 } } peers { permanent_uuid: "74eb6a2589da4205b5e5623842ef6faa" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 45549 } } peers { permanent_uuid: "18c08fd1a0d84a61ab7e6b98a82d3c11" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 45903 } }
I20260806 03:46:56.487109 15424 leader_election.cc:288] T 9ae270f9e8da4c02bf09c754a0d33304 P 74eb6a2589da4205b5e5623842ef6faa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2577e879d33e41ffb5a214cdae767802 (127.11.253.193:41343), 18c08fd1a0d84a61ab7e6b98a82d3c11 (127.11.253.195:45903)
I20260806 03:46:56.487296 15236 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ae270f9e8da4c02bf09c754a0d33304" candidate_uuid: "74eb6a2589da4205b5e5623842ef6faa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "18c08fd1a0d84a61ab7e6b98a82d3c11" is_pre_election: true
I20260806 03:46:56.487296 14837 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ae270f9e8da4c02bf09c754a0d33304" candidate_uuid: "74eb6a2589da4205b5e5623842ef6faa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2577e879d33e41ffb5a214cdae767802" is_pre_election: true
I20260806 03:46:56.487412 14837 raft_consensus.cc:2454] T 9ae270f9e8da4c02bf09c754a0d33304 P 2577e879d33e41ffb5a214cdae767802 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74eb6a2589da4205b5e5623842ef6faa in term 0.
I20260806 03:46:56.487412 15236 raft_consensus.cc:2454] T 9ae270f9e8da4c02bf09c754a0d33304 P 18c08fd1a0d84a61ab7e6b98a82d3c11 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74eb6a2589da4205b5e5623842ef6faa in term 0.
I20260806 03:46:56.487562 14935 leader_election.cc:302] T 9ae270f9e8da4c02bf09c754a0d33304 P 74eb6a2589da4205b5e5623842ef6faa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 18c08fd1a0d84a61ab7e6b98a82d3c11, 74eb6a2589da4205b5e5623842ef6faa; no voters:
I20260806 03:46:56.487700 15424 raft_consensus.cc:2781] T 9ae270f9e8da4c02bf09c754a0d33304 P 74eb6a2589da4205b5e5623842ef6faa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 03:46:56.487764 15424 raft_consensus.cc:482] T 9ae270f9e8da4c02bf09c754a0d33304 P 74eb6a2589da4205b5e5623842ef6faa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 03:46:56.487780 15424 raft_consensus.cc:3037] T 9ae270f9e8da4c02bf09c754a0d33304 P 74eb6a2589da4205b5e5623842ef6faa [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:46:56.488543 15424 raft_consensus.cc:504] T 9ae270f9e8da4c02bf09c754a0d33304 P 74eb6a2589da4205b5e5623842ef6faa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2577e879d33e41ffb5a214cdae767802" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 41343 } } peers { permanent_uuid: "74eb6a2589da4205b5e5623842ef6faa" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 45549 } } peers { permanent_uuid: "18c08fd1a0d84a61ab7e6b98a82d3c11" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 45903 } }
I20260806 03:46:56.488699 15424 leader_election.cc:288] T 9ae270f9e8da4c02bf09c754a0d33304 P 74eb6a2589da4205b5e5623842ef6faa [CANDIDATE]: Term 1 election: Requested vote from peers 2577e879d33e41ffb5a214cdae767802 (127.11.253.193:41343), 18c08fd1a0d84a61ab7e6b98a82d3c11 (127.11.253.195:45903)
I20260806 03:46:56.488880 14837 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ae270f9e8da4c02bf09c754a0d33304" candidate_uuid: "74eb6a2589da4205b5e5623842ef6faa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2577e879d33e41ffb5a214cdae767802"
I20260806 03:46:56.488943 15236 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ae270f9e8da4c02bf09c754a0d33304" candidate_uuid: "74eb6a2589da4205b5e5623842ef6faa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "18c08fd1a0d84a61ab7e6b98a82d3c11"
I20260806 03:46:56.488986 14837 raft_consensus.cc:3037] T 9ae270f9e8da4c02bf09c754a0d33304 P 2577e879d33e41ffb5a214cdae767802 [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:46:56.489013 15236 raft_consensus.cc:3037] T 9ae270f9e8da4c02bf09c754a0d33304 P 18c08fd1a0d84a61ab7e6b98a82d3c11 [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:46:56.489756 15236 raft_consensus.cc:2454] T 9ae270f9e8da4c02bf09c754a0d33304 P 18c08fd1a0d84a61ab7e6b98a82d3c11 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74eb6a2589da4205b5e5623842ef6faa in term 1.
I20260806 03:46:56.489756 14837 raft_consensus.cc:2454] T 9ae270f9e8da4c02bf09c754a0d33304 P 2577e879d33e41ffb5a214cdae767802 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74eb6a2589da4205b5e5623842ef6faa in term 1.
I20260806 03:46:56.489995 14938 leader_election.cc:302] T 9ae270f9e8da4c02bf09c754a0d33304 P 74eb6a2589da4205b5e5623842ef6faa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2577e879d33e41ffb5a214cdae767802, 74eb6a2589da4205b5e5623842ef6faa; no voters:
I20260806 03:46:56.490163 15424 raft_consensus.cc:2781] T 9ae270f9e8da4c02bf09c754a0d33304 P 74eb6a2589da4205b5e5623842ef6faa [term 1 FOLLOWER]: Leader election won for term 1
I20260806 03:46:56.490245 15424 raft_consensus.cc:686] T 9ae270f9e8da4c02bf09c754a0d33304 P 74eb6a2589da4205b5e5623842ef6faa [term 1 LEADER]: Becoming Leader. State: Replica: 74eb6a2589da4205b5e5623842ef6faa, State: Running, Role: LEADER
I20260806 03:46:56.490299 15424 consensus_queue.cc:237] T 9ae270f9e8da4c02bf09c754a0d33304 P 74eb6a2589da4205b5e5623842ef6faa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2577e879d33e41ffb5a214cdae767802" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 41343 } } peers { permanent_uuid: "74eb6a2589da4205b5e5623842ef6faa" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 45549 } } peers { permanent_uuid: "18c08fd1a0d84a61ab7e6b98a82d3c11" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 45903 } }
I20260806 03:46:56.490885 14606 catalog_manager.cc:5495] T 9ae270f9e8da4c02bf09c754a0d33304 P 74eb6a2589da4205b5e5623842ef6faa reported cstate change: term changed from 0 to 1, leader changed from <none> to 74eb6a2589da4205b5e5623842ef6faa (127.11.253.194). New cstate: current_term: 1 leader_uuid: "74eb6a2589da4205b5e5623842ef6faa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2577e879d33e41ffb5a214cdae767802" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 41343 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "74eb6a2589da4205b5e5623842ef6faa" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 45549 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "18c08fd1a0d84a61ab7e6b98a82d3c11" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 45903 } health_report { overall_health: UNKNOWN } } }
I20260806 03:46:56.498124 15477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:46:56.495898 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44532 (local address 127.11.253.194:45549)
0806 03:46:56.496049 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0806 03:46:56.496051 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:46:56.496060 (+ 9us) server_negotiation.cc:407] Connection header received
0806 03:46:56.496087 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:46:56.496088 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:46:56.496097 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:46:56.496157 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 03:46:56.496326 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:46:56.496777 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:46:56.497518 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:46:56.497698 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:46:56.497747 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:46:56.497903 (+ 156us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 03:46:56.497936 (+ 33us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 03:46:56.497988 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0806 03:46:56.498051 (+ 63us) server_negotiation.cc:299] Negotiation successful
0806 03:46:56.498069 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":106,"threads_started":1}
I20260806 03:46:56.500267 14837 raft_consensus.cc:1261] T 9ae270f9e8da4c02bf09c754a0d33304 P 2577e879d33e41ffb5a214cdae767802 [term 1 FOLLOWER]: Refusing update from remote peer 74eb6a2589da4205b5e5623842ef6faa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 03:46:56.500291 15236 raft_consensus.cc:1261] T 9ae270f9e8da4c02bf09c754a0d33304 P 18c08fd1a0d84a61ab7e6b98a82d3c11 [term 1 FOLLOWER]: Refusing update from remote peer 74eb6a2589da4205b5e5623842ef6faa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 03:46:56.500479 15405 consensus_queue.cc:1035] T 9ae270f9e8da4c02bf09c754a0d33304 P 74eb6a2589da4205b5e5623842ef6faa [LEADER]: Connected to new peer: Peer: permanent_uuid: "2577e879d33e41ffb5a214cdae767802" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 41343 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 03:46:56.500633 15481 consensus_queue.cc:1035] T 9ae270f9e8da4c02bf09c754a0d33304 P 74eb6a2589da4205b5e5623842ef6faa [LEADER]: Connected to new peer: Peer: permanent_uuid: "18c08fd1a0d84a61ab7e6b98a82d3c11" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 45903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 03:46:56.510767 14604 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60644:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 03:46:56.510874 14604 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60644:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 03:46:56.511525 14604 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ae915b1dc3a24bdb8afeed530e88edf7: Sending DeleteTablet for 3 replicas of tablet 9ae270f9e8da4c02bf09c754a0d33304
I20260806 03:46:56.511786 14801 tablet_service.cc:1514] Processing DeleteTablet for tablet 9ae270f9e8da4c02bf09c754a0d33304 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 03:46:56 UTC) from {username='kudu', principal='kudu/127.11.253.254@KRBTEST.COM'} at 127.0.0.1:55358
I20260806 03:46:56.511792 15002 tablet_service.cc:1514] Processing DeleteTablet for tablet 9ae270f9e8da4c02bf09c754a0d33304 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 03:46:56 UTC) from {username='kudu', principal='kudu/127.11.253.254@KRBTEST.COM'} at 127.0.0.1:44504
I20260806 03:46:56.511885 15444 tablet_replica.cc:330] T 9ae270f9e8da4c02bf09c754a0d33304 P 2577e879d33e41ffb5a214cdae767802: stopping tablet replica
I20260806 03:46:56.511934 15444 raft_consensus.cc:2229] T 9ae270f9e8da4c02bf09c754a0d33304 P 2577e879d33e41ffb5a214cdae767802 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 03:46:56.511900 15204 tablet_service.cc:1514] Processing DeleteTablet for tablet 9ae270f9e8da4c02bf09c754a0d33304 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 03:46:56 UTC) from {username='kudu', principal='kudu/127.11.253.254@KRBTEST.COM'} at 127.0.0.1:35384
I20260806 03:46:56.511996 15446 tablet_replica.cc:330] T 9ae270f9e8da4c02bf09c754a0d33304 P 18c08fd1a0d84a61ab7e6b98a82d3c11: stopping tablet replica
I20260806 03:46:56.512007 15444 raft_consensus.cc:2258] T 9ae270f9e8da4c02bf09c754a0d33304 P 2577e879d33e41ffb5a214cdae767802 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 03:46:56.512073 15446 raft_consensus.cc:2229] T 9ae270f9e8da4c02bf09c754a0d33304 P 18c08fd1a0d84a61ab7e6b98a82d3c11 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 03:46:56.512087 15442 tablet_replica.cc:330] T 9ae270f9e8da4c02bf09c754a0d33304 P 74eb6a2589da4205b5e5623842ef6faa: stopping tablet replica
I20260806 03:46:56.512034 14604 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60644:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260806 03:46:56.512121 15446 raft_consensus.cc:2258] T 9ae270f9e8da4c02bf09c754a0d33304 P 18c08fd1a0d84a61ab7e6b98a82d3c11 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 03:46:56.512141 15442 raft_consensus.cc:2229] T 9ae270f9e8da4c02bf09c754a0d33304 P 74eb6a2589da4205b5e5623842ef6faa [term 1 LEADER]: Raft consensus shutting down.
W20260806 03:46:56.512179 14604 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 03:46:56.512193 15442 raft_consensus.cc:2258] T 9ae270f9e8da4c02bf09c754a0d33304 P 74eb6a2589da4205b5e5623842ef6faa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 03:46:56.512399 15446 ts_tablet_manager.cc:1905] T 9ae270f9e8da4c02bf09c754a0d33304 P 18c08fd1a0d84a61ab7e6b98a82d3c11: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 03:46:56.512722 15444 ts_tablet_manager.cc:1905] T 9ae270f9e8da4c02bf09c754a0d33304 P 2577e879d33e41ffb5a214cdae767802: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 03:46:56.512833 15442 ts_tablet_manager.cc:1905] T 9ae270f9e8da4c02bf09c754a0d33304 P 74eb6a2589da4205b5e5623842ef6faa: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 03:46:56.514191 15446 ts_tablet_manager.cc:1918] T 9ae270f9e8da4c02bf09c754a0d33304 P 18c08fd1a0d84a61ab7e6b98a82d3c11: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 03:46:56.514235 15446 log.cc:1196] T 9ae270f9e8da4c02bf09c754a0d33304 P 18c08fd1a0d84a61ab7e6b98a82d3c11: Deleting WAL directory at /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785988004637995-12279-0/minicluster-data/ts-2/wal/wals/9ae270f9e8da4c02bf09c754a0d33304
I20260806 03:46:56.514452 15442 ts_tablet_manager.cc:1918] T 9ae270f9e8da4c02bf09c754a0d33304 P 74eb6a2589da4205b5e5623842ef6faa: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 03:46:56.514514 15442 log.cc:1196] T 9ae270f9e8da4c02bf09c754a0d33304 P 74eb6a2589da4205b5e5623842ef6faa: Deleting WAL directory at /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785988004637995-12279-0/minicluster-data/ts-1/wal/wals/9ae270f9e8da4c02bf09c754a0d33304
I20260806 03:46:56.514513 15446 ts_tablet_manager.cc:1939] T 9ae270f9e8da4c02bf09c754a0d33304 P 18c08fd1a0d84a61ab7e6b98a82d3c11: Deleting consensus metadata
I20260806 03:46:56.514617 15444 ts_tablet_manager.cc:1918] T 9ae270f9e8da4c02bf09c754a0d33304 P 2577e879d33e41ffb5a214cdae767802: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 03:46:56.514680 15444 log.cc:1196] T 9ae270f9e8da4c02bf09c754a0d33304 P 2577e879d33e41ffb5a214cdae767802: Deleting WAL directory at /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785988004637995-12279-0/minicluster-data/ts-0/wal/wals/9ae270f9e8da4c02bf09c754a0d33304
I20260806 03:46:56.514750 15442 ts_tablet_manager.cc:1939] T 9ae270f9e8da4c02bf09c754a0d33304 P 74eb6a2589da4205b5e5623842ef6faa: Deleting consensus metadata
I20260806 03:46:56.514910 15444 ts_tablet_manager.cc:1939] T 9ae270f9e8da4c02bf09c754a0d33304 P 2577e879d33e41ffb5a214cdae767802: Deleting consensus metadata
I20260806 03:46:56.515027 14588 catalog_manager.cc:4841] TS 18c08fd1a0d84a61ab7e6b98a82d3c11 (127.11.253.195:45903): tablet 9ae270f9e8da4c02bf09c754a0d33304 (table test-table [id=4461443b4b5b45e7a70c2bd95dd48f67]) successfully deleted
I20260806 03:46:56.515249 14590 catalog_manager.cc:4841] TS 74eb6a2589da4205b5e5623842ef6faa (127.11.253.194:45549): tablet 9ae270f9e8da4c02bf09c754a0d33304 (table test-table [id=4461443b4b5b45e7a70c2bd95dd48f67]) successfully deleted
I20260806 03:46:56.515506 15002 tablet_service.cc:1467] Processing CreateTablet for tablet 258b9a18af7e40858833906cbc53bd37 (DEFAULT_TABLE table=test-table [id=b115ce612148475a8fb84281701cca73]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 03:46:56.515511 15204 tablet_service.cc:1467] Processing CreateTablet for tablet 258b9a18af7e40858833906cbc53bd37 (DEFAULT_TABLE table=test-table [id=b115ce612148475a8fb84281701cca73]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 03:46:56.515537 14801 tablet_service.cc:1467] Processing CreateTablet for tablet 258b9a18af7e40858833906cbc53bd37 (DEFAULT_TABLE table=test-table [id=b115ce612148475a8fb84281701cca73]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 03:46:56.515614 15002 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 258b9a18af7e40858833906cbc53bd37. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:46:56.515621 14801 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 258b9a18af7e40858833906cbc53bd37. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:46:56.515616 15204 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 258b9a18af7e40858833906cbc53bd37. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:46:56.516503 15465 tablet_bootstrap.cc:492] T 258b9a18af7e40858833906cbc53bd37 P 18c08fd1a0d84a61ab7e6b98a82d3c11: Bootstrap starting.
I20260806 03:46:56.516795 15466 tablet_bootstrap.cc:492] T 258b9a18af7e40858833906cbc53bd37 P 2577e879d33e41ffb5a214cdae767802: Bootstrap starting.
I20260806 03:46:56.516896 15465 tablet_bootstrap.cc:654] T 258b9a18af7e40858833906cbc53bd37 P 18c08fd1a0d84a61ab7e6b98a82d3c11: Neither blocks nor log segments found. Creating new log.
I20260806 03:46:56.517239 15466 tablet_bootstrap.cc:654] T 258b9a18af7e40858833906cbc53bd37 P 2577e879d33e41ffb5a214cdae767802: Neither blocks nor log segments found. Creating new log.
I20260806 03:46:56.517530 15465 tablet_bootstrap.cc:492] T 258b9a18af7e40858833906cbc53bd37 P 18c08fd1a0d84a61ab7e6b98a82d3c11: No bootstrap required, opened a new log
I20260806 03:46:56.517582 15465 ts_tablet_manager.cc:1397] T 258b9a18af7e40858833906cbc53bd37 P 18c08fd1a0d84a61ab7e6b98a82d3c11: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260806 03:46:56.517603 15462 tablet_bootstrap.cc:492] T 258b9a18af7e40858833906cbc53bd37 P 74eb6a2589da4205b5e5623842ef6faa: Bootstrap starting.
I20260806 03:46:56.517704 15465 raft_consensus.cc:348] T 258b9a18af7e40858833906cbc53bd37 P 18c08fd1a0d84a61ab7e6b98a82d3c11 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2577e879d33e41ffb5a214cdae767802" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 41343 } } peers { permanent_uuid: "74eb6a2589da4205b5e5623842ef6faa" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 45549 } } peers { permanent_uuid: "18c08fd1a0d84a61ab7e6b98a82d3c11" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 45903 } }
I20260806 03:46:56.517800 15465 raft_consensus.cc:374] T 258b9a18af7e40858833906cbc53bd37 P 18c08fd1a0d84a61ab7e6b98a82d3c11 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:46:56.517828 15465 raft_consensus.cc:729] T 258b9a18af7e40858833906cbc53bd37 P 18c08fd1a0d84a61ab7e6b98a82d3c11 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 18c08fd1a0d84a61ab7e6b98a82d3c11, State: Initialized, Role: FOLLOWER
I20260806 03:46:56.517928 15465 consensus_queue.cc:260] T 258b9a18af7e40858833906cbc53bd37 P 18c08fd1a0d84a61ab7e6b98a82d3c11 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2577e879d33e41ffb5a214cdae767802" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 41343 } } peers { permanent_uuid: "74eb6a2589da4205b5e5623842ef6faa" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 45549 } } peers { permanent_uuid: "18c08fd1a0d84a61ab7e6b98a82d3c11" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 45903 } }
I20260806 03:46:56.517975 15462 tablet_bootstrap.cc:654] T 258b9a18af7e40858833906cbc53bd37 P 74eb6a2589da4205b5e5623842ef6faa: Neither blocks nor log segments found. Creating new log.
I20260806 03:46:56.518046 15465 ts_tablet_manager.cc:1428] T 258b9a18af7e40858833906cbc53bd37 P 18c08fd1a0d84a61ab7e6b98a82d3c11: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260806 03:46:56.518105 15466 tablet_bootstrap.cc:492] T 258b9a18af7e40858833906cbc53bd37 P 2577e879d33e41ffb5a214cdae767802: No bootstrap required, opened a new log
I20260806 03:46:56.518153 15466 ts_tablet_manager.cc:1397] T 258b9a18af7e40858833906cbc53bd37 P 2577e879d33e41ffb5a214cdae767802: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260806 03:46:56.518307 15466 raft_consensus.cc:348] T 258b9a18af7e40858833906cbc53bd37 P 2577e879d33e41ffb5a214cdae767802 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2577e879d33e41ffb5a214cdae767802" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 41343 } } peers { permanent_uuid: "74eb6a2589da4205b5e5623842ef6faa" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 45549 } } peers { permanent_uuid: "18c08fd1a0d84a61ab7e6b98a82d3c11" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 45903 } }
I20260806 03:46:56.518368 15466 raft_consensus.cc:374] T 258b9a18af7e40858833906cbc53bd37 P 2577e879d33e41ffb5a214cdae767802 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:46:56.518386 15466 raft_consensus.cc:729] T 258b9a18af7e40858833906cbc53bd37 P 2577e879d33e41ffb5a214cdae767802 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2577e879d33e41ffb5a214cdae767802, State: Initialized, Role: FOLLOWER
I20260806 03:46:56.518433 15466 consensus_queue.cc:260] T 258b9a18af7e40858833906cbc53bd37 P 2577e879d33e41ffb5a214cdae767802 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2577e879d33e41ffb5a214cdae767802" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 41343 } } peers { permanent_uuid: "74eb6a2589da4205b5e5623842ef6faa" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 45549 } } peers { permanent_uuid: "18c08fd1a0d84a61ab7e6b98a82d3c11" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 45903 } }
I20260806 03:46:56.518512 15466 ts_tablet_manager.cc:1428] T 258b9a18af7e40858833906cbc53bd37 P 2577e879d33e41ffb5a214cdae767802: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 03:46:56.519088 14591 catalog_manager.cc:4841] TS 2577e879d33e41ffb5a214cdae767802 (127.11.253.193:41343): tablet 9ae270f9e8da4c02bf09c754a0d33304 (table test-table [id=4461443b4b5b45e7a70c2bd95dd48f67]) successfully deleted
I20260806 03:46:56.519675 15462 tablet_bootstrap.cc:492] T 258b9a18af7e40858833906cbc53bd37 P 74eb6a2589da4205b5e5623842ef6faa: No bootstrap required, opened a new log
I20260806 03:46:56.519721 15462 ts_tablet_manager.cc:1397] T 258b9a18af7e40858833906cbc53bd37 P 74eb6a2589da4205b5e5623842ef6faa: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260806 03:46:56.519852 15462 raft_consensus.cc:348] T 258b9a18af7e40858833906cbc53bd37 P 74eb6a2589da4205b5e5623842ef6faa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2577e879d33e41ffb5a214cdae767802" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 41343 } } peers { permanent_uuid: "74eb6a2589da4205b5e5623842ef6faa" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 45549 } } peers { permanent_uuid: "18c08fd1a0d84a61ab7e6b98a82d3c11" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 45903 } }
I20260806 03:46:56.519932 15462 raft_consensus.cc:374] T 258b9a18af7e40858833906cbc53bd37 P 74eb6a2589da4205b5e5623842ef6faa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:46:56.519953 15462 raft_consensus.cc:729] T 258b9a18af7e40858833906cbc53bd37 P 74eb6a2589da4205b5e5623842ef6faa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74eb6a2589da4205b5e5623842ef6faa, State: Initialized, Role: FOLLOWER
I20260806 03:46:56.519994 15462 consensus_queue.cc:260] T 258b9a18af7e40858833906cbc53bd37 P 74eb6a2589da4205b5e5623842ef6faa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2577e879d33e41ffb5a214cdae767802" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 41343 } } peers { permanent_uuid: "74eb6a2589da4205b5e5623842ef6faa" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 45549 } } peers { permanent_uuid: "18c08fd1a0d84a61ab7e6b98a82d3c11" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 45903 } }
I20260806 03:46:56.520090 15462 ts_tablet_manager.cc:1428] T 258b9a18af7e40858833906cbc53bd37 P 74eb6a2589da4205b5e5623842ef6faa: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 03:46:56.553828 15471 raft_consensus.cc:482] T 258b9a18af7e40858833906cbc53bd37 P 2577e879d33e41ffb5a214cdae767802 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 03:46:56.554018 15471 raft_consensus.cc:504] T 258b9a18af7e40858833906cbc53bd37 P 2577e879d33e41ffb5a214cdae767802 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2577e879d33e41ffb5a214cdae767802" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 41343 } } peers { permanent_uuid: "74eb6a2589da4205b5e5623842ef6faa" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 45549 } } peers { permanent_uuid: "18c08fd1a0d84a61ab7e6b98a82d3c11" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 45903 } }
I20260806 03:46:56.554539 15471 leader_election.cc:288] T 258b9a18af7e40858833906cbc53bd37 P 2577e879d33e41ffb5a214cdae767802 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 74eb6a2589da4205b5e5623842ef6faa (127.11.253.194:45549), 18c08fd1a0d84a61ab7e6b98a82d3c11 (127.11.253.195:45903)
I20260806 03:46:56.557612 15490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:46:56.554761 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.194:45549 (local address 127.11.253.193:51521)
0806 03:46:56.554916 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0806 03:46:56.554929 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 03:46:56.554989 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:46:56.555151 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:46:56.555152 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:46:56.555154 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 03:46:56.555239 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:46:56.555240 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:46:56.555938 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:46:56.555939 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:46:56.556778 (+ 839us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:46:56.556780 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:46:56.557334 (+ 554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:46:56.557335 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:46:56.557413 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:46:56.557416 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 03:46:56.557482 (+ 66us) client_negotiation.cc:240] Negotiation successful
0806 03:46:56.557524 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":85,"threads_started":1}
I20260806 03:46:56.557598 15477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:46:56.554935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.253.193:51521 (local address 127.11.253.194:45549)
0806 03:46:56.554983 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0806 03:46:56.554985 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:46:56.554994 (+ 9us) server_negotiation.cc:407] Connection header received
0806 03:46:56.555061 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:46:56.555062 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:46:56.555071 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:46:56.555148 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 03:46:56.555364 (+ 216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:46:56.555854 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:46:56.556874 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:46:56.557249 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:46:56.557336 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:46:56.557347 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0806 03:46:56.557488 (+ 141us) server_negotiation.cc:299] Negotiation successful
0806 03:46:56.557531 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260806 03:46:56.557807 15492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:46:56.554841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.253.193:56629 (local address 127.11.253.195:45903)
0806 03:46:56.554967 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0806 03:46:56.554969 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:46:56.555042 (+ 73us) server_negotiation.cc:407] Connection header received
0806 03:46:56.555112 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:46:56.555113 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:46:56.555126 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:46:56.555207 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 03:46:56.555402 (+ 195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:46:56.555928 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:46:56.556964 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:46:56.557400 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:46:56.557498 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:46:56.557507 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0806 03:46:56.557669 (+ 162us) server_negotiation.cc:299] Negotiation successful
0806 03:46:56.557718 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104,"thread_start_us":54,"threads_started":1}
I20260806 03:46:56.557806 15491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:46:56.554769 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.195:45903 (local address 127.11.253.193:56629)
0806 03:46:56.554951 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0806 03:46:56.554962 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 03:46:56.555037 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:46:56.555214 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:46:56.555214 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:46:56.555217 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 03:46:56.555311 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:46:56.555313 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:46:56.556040 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:46:56.556041 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:46:56.556863 (+ 822us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:46:56.556865 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:46:56.557495 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:46:56.557496 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:46:56.557585 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:46:56.557588 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 03:46:56.557664 (+ 76us) client_negotiation.cc:240] Negotiation successful
0806 03:46:56.557706 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":109,"threads_started":1}
I20260806 03:46:56.557839 15037 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "258b9a18af7e40858833906cbc53bd37" candidate_uuid: "2577e879d33e41ffb5a214cdae767802" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74eb6a2589da4205b5e5623842ef6faa" is_pre_election: true
I20260806 03:46:56.557960 15037 raft_consensus.cc:2454] T 258b9a18af7e40858833906cbc53bd37 P 74eb6a2589da4205b5e5623842ef6faa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2577e879d33e41ffb5a214cdae767802 in term 0.
I20260806 03:46:56.558065 15236 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "258b9a18af7e40858833906cbc53bd37" candidate_uuid: "2577e879d33e41ffb5a214cdae767802" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "18c08fd1a0d84a61ab7e6b98a82d3c11" is_pre_election: true
I20260806 03:46:56.558162 15236 raft_consensus.cc:2454] T 258b9a18af7e40858833906cbc53bd37 P 18c08fd1a0d84a61ab7e6b98a82d3c11 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2577e879d33e41ffb5a214cdae767802 in term 0.
I20260806 03:46:56.558139 14737 leader_election.cc:302] T 258b9a18af7e40858833906cbc53bd37 P 2577e879d33e41ffb5a214cdae767802 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2577e879d33e41ffb5a214cdae767802, 74eb6a2589da4205b5e5623842ef6faa; no voters:
I20260806 03:46:56.558257 15471 raft_consensus.cc:2781] T 258b9a18af7e40858833906cbc53bd37 P 2577e879d33e41ffb5a214cdae767802 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 03:46:56.558298 15471 raft_consensus.cc:482] T 258b9a18af7e40858833906cbc53bd37 P 2577e879d33e41ffb5a214cdae767802 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 03:46:56.558327 15471 raft_consensus.cc:3037] T 258b9a18af7e40858833906cbc53bd37 P 2577e879d33e41ffb5a214cdae767802 [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:46:56.559031 15471 raft_consensus.cc:504] T 258b9a18af7e40858833906cbc53bd37 P 2577e879d33e41ffb5a214cdae767802 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2577e879d33e41ffb5a214cdae767802" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 41343 } } peers { permanent_uuid: "74eb6a2589da4205b5e5623842ef6faa" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 45549 } } peers { permanent_uuid: "18c08fd1a0d84a61ab7e6b98a82d3c11" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 45903 } }
I20260806 03:46:56.559185 15471 leader_election.cc:288] T 258b9a18af7e40858833906cbc53bd37 P 2577e879d33e41ffb5a214cdae767802 [CANDIDATE]: Term 1 election: Requested vote from peers 74eb6a2589da4205b5e5623842ef6faa (127.11.253.194:45549), 18c08fd1a0d84a61ab7e6b98a82d3c11 (127.11.253.195:45903)
I20260806 03:46:56.559335 15236 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "258b9a18af7e40858833906cbc53bd37" candidate_uuid: "2577e879d33e41ffb5a214cdae767802" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "18c08fd1a0d84a61ab7e6b98a82d3c11"
I20260806 03:46:56.559440 15236 raft_consensus.cc:3037] T 258b9a18af7e40858833906cbc53bd37 P 18c08fd1a0d84a61ab7e6b98a82d3c11 [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:46:56.559429 15037 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "258b9a18af7e40858833906cbc53bd37" candidate_uuid: "2577e879d33e41ffb5a214cdae767802" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74eb6a2589da4205b5e5623842ef6faa"
I20260806 03:46:56.559504 15037 raft_consensus.cc:3037] T 258b9a18af7e40858833906cbc53bd37 P 74eb6a2589da4205b5e5623842ef6faa [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:46:56.560235 15236 raft_consensus.cc:2454] T 258b9a18af7e40858833906cbc53bd37 P 18c08fd1a0d84a61ab7e6b98a82d3c11 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2577e879d33e41ffb5a214cdae767802 in term 1.
I20260806 03:46:56.560235 15037 raft_consensus.cc:2454] T 258b9a18af7e40858833906cbc53bd37 P 74eb6a2589da4205b5e5623842ef6faa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2577e879d33e41ffb5a214cdae767802 in term 1.
I20260806 03:46:56.560436 14737 leader_election.cc:302] T 258b9a18af7e40858833906cbc53bd37 P 2577e879d33e41ffb5a214cdae767802 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2577e879d33e41ffb5a214cdae767802, 74eb6a2589da4205b5e5623842ef6faa; no voters:
I20260806 03:46:56.560600 15471 raft_consensus.cc:2781] T 258b9a18af7e40858833906cbc53bd37 P 2577e879d33e41ffb5a214cdae767802 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 03:46:56.560788 15471 raft_consensus.cc:686] T 258b9a18af7e40858833906cbc53bd37 P 2577e879d33e41ffb5a214cdae767802 [term 1 LEADER]: Becoming Leader. State: Replica: 2577e879d33e41ffb5a214cdae767802, State: Running, Role: LEADER
I20260806 03:46:56.560878 15471 consensus_queue.cc:237] T 258b9a18af7e40858833906cbc53bd37 P 2577e879d33e41ffb5a214cdae767802 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2577e879d33e41ffb5a214cdae767802" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 41343 } } peers { permanent_uuid: "74eb6a2589da4205b5e5623842ef6faa" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 45549 } } peers { permanent_uuid: "18c08fd1a0d84a61ab7e6b98a82d3c11" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 45903 } }
I20260806 03:46:56.561635 14604 catalog_manager.cc:5495] T 258b9a18af7e40858833906cbc53bd37 P 2577e879d33e41ffb5a214cdae767802 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2577e879d33e41ffb5a214cdae767802 (127.11.253.193). New cstate: current_term: 1 leader_uuid: "2577e879d33e41ffb5a214cdae767802" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2577e879d33e41ffb5a214cdae767802" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 41343 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "74eb6a2589da4205b5e5623842ef6faa" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 45549 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "18c08fd1a0d84a61ab7e6b98a82d3c11" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 45903 } health_report { overall_health: UNKNOWN } } }
I20260806 03:46:56.569393 15496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:46:56.566969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55364 (local address 127.11.253.193:41343)
0806 03:46:56.567119 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0806 03:46:56.567121 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:46:56.567127 (+ 6us) server_negotiation.cc:407] Connection header received
0806 03:46:56.567158 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:46:56.567158 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:46:56.567175 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:46:56.567244 (+ 69us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 03:46:56.567359 (+ 115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:46:56.567817 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:46:56.568535 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:46:56.568780 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:46:56.568860 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:46:56.569028 (+ 168us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 03:46:56.569146 (+ 118us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 03:46:56.569231 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0806 03:46:56.569276 (+ 45us) server_negotiation.cc:299] Negotiation successful
0806 03:46:56.569296 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":90,"threads_started":1}
I20260806 03:46:56.574265 15477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:46:56.571957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.253.193:56271 (local address 127.11.253.194:45549)
0806 03:46:56.571997 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0806 03:46:56.571999 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:46:56.572099 (+ 100us) server_negotiation.cc:407] Connection header received
0806 03:46:56.572128 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:46:56.572128 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:46:56.572137 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:46:56.572233 (+ 96us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 03:46:56.572284 (+ 51us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:46:56.572820 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:46:56.573571 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:46:56.573776 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:46:56.573837 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:46:56.574014 (+ 177us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 03:46:56.574044 (+ 30us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 03:46:56.574091 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0806 03:46:56.574185 (+ 94us) server_negotiation.cc:299] Negotiation successful
0806 03:46:56.574210 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 03:46:56.574306 15499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:46:56.571872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.194:45549 (local address 127.11.253.193:56271)
0806 03:46:56.572022 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0806 03:46:56.572032 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 03:46:56.572093 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:46:56.572190 (+ 97us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:46:56.572191 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:46:56.572192 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 03:46:56.572241 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:46:56.572242 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:46:56.572895 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:46:56.572896 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:46:56.573488 (+ 592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:46:56.573490 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:46:56.573883 (+ 393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:46:56.573884 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:46:56.573927 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:46:56.573942 (+ 15us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 03:46:56.574134 (+ 192us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 03:46:56.574137 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 03:46:56.574183 (+ 46us) client_negotiation.cc:240] Negotiation successful
0806 03:46:56.574220 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":78,"threads_started":1}
I20260806 03:46:56.576702 15037 raft_consensus.cc:1261] T 258b9a18af7e40858833906cbc53bd37 P 74eb6a2589da4205b5e5623842ef6faa [term 1 FOLLOWER]: Refusing update from remote peer 2577e879d33e41ffb5a214cdae767802: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 03:46:56.576735 15236 raft_consensus.cc:1261] T 258b9a18af7e40858833906cbc53bd37 P 18c08fd1a0d84a61ab7e6b98a82d3c11 [term 1 FOLLOWER]: Refusing update from remote peer 2577e879d33e41ffb5a214cdae767802: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 03:46:56.576985 15471 consensus_queue.cc:1035] T 258b9a18af7e40858833906cbc53bd37 P 2577e879d33e41ffb5a214cdae767802 [LEADER]: Connected to new peer: Peer: permanent_uuid: "18c08fd1a0d84a61ab7e6b98a82d3c11" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 45903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 03:46:56.577134 15493 consensus_queue.cc:1035] T 258b9a18af7e40858833906cbc53bd37 P 2577e879d33e41ffb5a214cdae767802 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74eb6a2589da4205b5e5623842ef6faa" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 45549 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 03:46:56.584652 15496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:46:56.582199 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.253.194:33045 (local address 127.11.253.193:41343)
0806 03:46:56.582238 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0806 03:46:56.582240 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:46:56.582422 (+ 182us) server_negotiation.cc:407] Connection header received
0806 03:46:56.582456 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:46:56.582457 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:46:56.582469 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:46:56.582525 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 03:46:56.582621 (+ 96us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:46:56.583063 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:46:56.583922 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:46:56.584226 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:46:56.584327 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:46:56.584493 (+ 166us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 03:46:56.584519 (+ 26us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 03:46:56.584555 (+ 36us) server_negotiation.cc:1035] Waiting for connection context
0806 03:46:56.584587 (+ 32us) server_negotiation.cc:299] Negotiation successful
0806 03:46:56.584603 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260806 03:46:56.584676 15507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:46:56.582182 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.193:41343 (local address 127.11.253.194:33045)
0806 03:46:56.582325 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0806 03:46:56.582352 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0806 03:46:56.582414 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:46:56.582525 (+ 111us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:46:56.582525 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:46:56.582527 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 03:46:56.582573 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:46:56.582574 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:46:56.583123 (+ 549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:46:56.583123 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:46:56.583801 (+ 678us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:46:56.583804 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:46:56.584339 (+ 535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:46:56.584340 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:46:56.584424 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:46:56.584430 (+ 6us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 03:46:56.584555 (+ 125us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 03:46:56.584556 (+ 1us) client_negotiation.cc:769] Sending connection context
0806 03:46:56.584585 (+ 29us) client_negotiation.cc:240] Negotiation successful
0806 03:46:56.584620 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":94,"threads_started":1}
W20260806 03:46:56.585152 15417 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260806 03:46:58.111064 14605 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60644:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 03:46:58.111173 14605 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60644:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 03:46:58.112151 14605 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ae915b1dc3a24bdb8afeed530e88edf7: Sending DeleteTablet for 3 replicas of tablet 258b9a18af7e40858833906cbc53bd37
I20260806 03:46:58.112448 14801 tablet_service.cc:1514] Processing DeleteTablet for tablet 258b9a18af7e40858833906cbc53bd37 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 03:46:58 UTC) from {username='kudu', principal='kudu/127.11.253.254@KRBTEST.COM'} at 127.0.0.1:55358
I20260806 03:46:58.112488 15204 tablet_service.cc:1514] Processing DeleteTablet for tablet 258b9a18af7e40858833906cbc53bd37 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 03:46:58 UTC) from {username='kudu', principal='kudu/127.11.253.254@KRBTEST.COM'} at 127.0.0.1:35384
I20260806 03:46:58.112577 12279 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskToVWRe/build/release/bin/kudu with pid 14701
I20260806 03:46:58.112670 15002 tablet_service.cc:1514] Processing DeleteTablet for tablet 258b9a18af7e40858833906cbc53bd37 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 03:46:58 UTC) from {username='kudu', principal='kudu/127.11.253.254@KRBTEST.COM'} at 127.0.0.1:44504
I20260806 03:46:58.120548 15538 tablet_replica.cc:330] T 258b9a18af7e40858833906cbc53bd37 P 74eb6a2589da4205b5e5623842ef6faa: stopping tablet replica
I20260806 03:46:58.120656 15538 raft_consensus.cc:2229] T 258b9a18af7e40858833906cbc53bd37 P 74eb6a2589da4205b5e5623842ef6faa [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 03:46:58.120721 15538 raft_consensus.cc:2258] T 258b9a18af7e40858833906cbc53bd37 P 74eb6a2589da4205b5e5623842ef6faa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 03:46:58.120869 15537 tablet_replica.cc:330] T 258b9a18af7e40858833906cbc53bd37 P 18c08fd1a0d84a61ab7e6b98a82d3c11: stopping tablet replica
I20260806 03:46:58.120973 15537 raft_consensus.cc:2229] T 258b9a18af7e40858833906cbc53bd37 P 18c08fd1a0d84a61ab7e6b98a82d3c11 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 03:46:58.121028 15537 raft_consensus.cc:2258] T 258b9a18af7e40858833906cbc53bd37 P 18c08fd1a0d84a61ab7e6b98a82d3c11 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 03:46:58.121042 15538 ts_tablet_manager.cc:1905] T 258b9a18af7e40858833906cbc53bd37 P 74eb6a2589da4205b5e5623842ef6faa: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 03:46:58.121284 15537 ts_tablet_manager.cc:1905] T 258b9a18af7e40858833906cbc53bd37 P 18c08fd1a0d84a61ab7e6b98a82d3c11: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 03:46:58.122989 15537 ts_tablet_manager.cc:1918] T 258b9a18af7e40858833906cbc53bd37 P 18c08fd1a0d84a61ab7e6b98a82d3c11: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 03:46:58.123009 15538 ts_tablet_manager.cc:1918] T 258b9a18af7e40858833906cbc53bd37 P 74eb6a2589da4205b5e5623842ef6faa: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 03:46:58.123061 15537 log.cc:1196] T 258b9a18af7e40858833906cbc53bd37 P 18c08fd1a0d84a61ab7e6b98a82d3c11: Deleting WAL directory at /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785988004637995-12279-0/minicluster-data/ts-2/wal/wals/258b9a18af7e40858833906cbc53bd37
I20260806 03:46:58.123061 15538 log.cc:1196] T 258b9a18af7e40858833906cbc53bd37 P 74eb6a2589da4205b5e5623842ef6faa: Deleting WAL directory at /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785988004637995-12279-0/minicluster-data/ts-1/wal/wals/258b9a18af7e40858833906cbc53bd37
I20260806 03:46:58.123301 15537 ts_tablet_manager.cc:1939] T 258b9a18af7e40858833906cbc53bd37 P 18c08fd1a0d84a61ab7e6b98a82d3c11: Deleting consensus metadata
I20260806 03:46:58.123306 15538 ts_tablet_manager.cc:1939] T 258b9a18af7e40858833906cbc53bd37 P 74eb6a2589da4205b5e5623842ef6faa: Deleting consensus metadata
I20260806 03:46:58.123660 14588 catalog_manager.cc:4841] TS 18c08fd1a0d84a61ab7e6b98a82d3c11 (127.11.253.195:45903): tablet 258b9a18af7e40858833906cbc53bd37 (table test-table [id=b115ce612148475a8fb84281701cca73]) successfully deleted
I20260806 03:46:58.123688 14590 catalog_manager.cc:4841] TS 74eb6a2589da4205b5e5623842ef6faa (127.11.253.194:45549): tablet 258b9a18af7e40858833906cbc53bd37 (table test-table [id=b115ce612148475a8fb84281701cca73]) successfully deleted
W20260806 03:46:58.128299 14591 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.253.193:41343 (error 108)
I20260806 03:46:58.128638 12279 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskToVWRe/build/release/bin/kudu with pid 14906
I20260806 03:46:58.133332 15540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:46:58.128742 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.193:41343 (local address 127.0.0.1:55368)
0806 03:46:58.133240 (+ 4498us) negotiation.cc:107] Waiting for socket to connect
0806 03:46:58.133293 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.253.193:41343: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":4472,"thread_start_us":64,"threads_started":1}
W20260806 03:46:58.133513 14591 catalog_manager.cc:4584] TS 2577e879d33e41ffb5a214cdae767802 (127.11.253.193:41343): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 258b9a18af7e40858833906cbc53bd37: Network error: Client connection negotiation failed: client connection to 127.11.253.193:41343: connect: Connection refused (error 111)
I20260806 03:46:58.141076 12279 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskToVWRe/build/release/bin/kudu with pid 15116
I20260806 03:46:58.151614 12279 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskToVWRe/build/release/bin/kudu with pid 14566
2026-08-06T03:46:58Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (6164 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785988004637995-12279-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 03:46:58 dist-test-slave-6ntk krb5kdc[15543](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 03:46:58 dist-test-slave-6ntk krb5kdc[15543](info): set up 2 sockets
Aug 06 03:46:58 dist-test-slave-6ntk krb5kdc[15543](info): commencing operation
krb5kdc: starting...
W20260806 03:46:59.375561 12279 mini_kdc.cc:121] Time spent starting KDC: real 1.196s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785988004637995-12279-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785988004637995-12279-0/krb5kdc/test-user.keytab.
Aug 06 03:46:59 dist-test-slave-6ntk krb5kdc[15543](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988019, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T03:46:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T03:46:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.253.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.253.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.253.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.254.keytab.
Entry for principal kudu/127.11.253.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.254.keytab.
I20260806 03:46:59.509822 12279 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskToVWRe/build/release/bin/kudu
/tmp/dist-test-taskToVWRe/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785988004637995-12279-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785988004637995-12279-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785988004637995-12279-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785988004637995-12279-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.253.254:34329
--webserver_interface=127.11.253.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.254.keytab
--principal=kudu/127.11.253.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.253.212:45223
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.253.254:34329
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785988004637995-12279-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785988004637995-12279-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785988004637995-12279-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:46:59.594249 15559 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:46:59.594460 15559 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:46:59.594487 15559 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:46:59.595714 15559 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 03:46:59.595759 15559 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:46:59.595773 15559 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:46:59.595786 15559 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 03:46:59.595798 15559 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 03:46:59.597317 15559 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.253.212:45223
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785988004637995-12279-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785988004637995-12279-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.253.254:34329
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.254.keytab
--principal=kudu/127.11.253.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.253.254:34329
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785988004637995-12279-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.253.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15559
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785988004637995-12279-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:46:59.597579 15559 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:46:59.597829 15559 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:46:59.603102 15565 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:46:59.603157 15566 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:46:59.603155 15569 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:46:59.603169 15559 server_base.cc:1034] running on GCE node
I20260806 03:46:59.603603 15559 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:46:59.603874 15559 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:46:59.605039 15559 hybrid_clock.cc:648] HybridClock initialized: now 1785988019605030 us; error 32 us; skew 500 ppm
Aug 06 03:46:59 dist-test-slave-6ntk krb5kdc[15543](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988019, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.253.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:46:59.607868 15559 init.cc:377] Logged in from keytab as kudu/127.11.253.254@KRBTEST.COM (short username kudu)
I20260806 03:46:59.608337 15559 webserver.cc:460] Webserver started at http://127.11.253.254:39385/ using document root <none> and password file <none>
I20260806 03:46:59.608594 15559 fs_manager.cc:359] Metadata directory not provided
I20260806 03:46:59.608640 15559 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:46:59.608826 15559 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:46:59.610049 15559 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785988004637995-12279-0/minicluster-data/master-0/data/instance:
uuid: "5c7a452da8e24f42a31e623986fcef71"
format_stamp: "Formatted at 2026-08-06 03:46:59 on dist-test-slave-6ntk"
server_key: "4c6b6cbbab29860c15ec88fafd9adc0e"
server_key_iv: "eeb2ad1447c639585b56b62f1eb18c3e"
server_key_version: "encryptionkey@0"
I20260806 03:46:59.610414 15559 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785988004637995-12279-0/minicluster-data/master-0/wal/instance:
uuid: "5c7a452da8e24f42a31e623986fcef71"
format_stamp: "Formatted at 2026-08-06 03:46:59 on dist-test-slave-6ntk"
server_key: "4c6b6cbbab29860c15ec88fafd9adc0e"
server_key_iv: "eeb2ad1447c639585b56b62f1eb18c3e"
server_key_version: "encryptionkey@0"
I20260806 03:46:59.611862 15559 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260806 03:46:59.612694 15578 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:46:59.612952 15559 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 03:46:59.613057 15559 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785988004637995-12279-0/minicluster-data/master-0/data,/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785988004637995-12279-0/minicluster-data/master-0/wal
uuid: "5c7a452da8e24f42a31e623986fcef71"
format_stamp: "Formatted at 2026-08-06 03:46:59 on dist-test-slave-6ntk"
server_key: "4c6b6cbbab29860c15ec88fafd9adc0e"
server_key_iv: "eeb2ad1447c639585b56b62f1eb18c3e"
server_key_version: "encryptionkey@0"
I20260806 03:46:59.613137 15559 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785988004637995-12279-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785988004637995-12279-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785988004637995-12279-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:46:59.622284 15559 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:46:59.624166 15559 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:46:59.624325 15559 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:46:59.628496 15559 rpc_server.cc:307] RPC server started. Bound to: 127.11.253.254:34329
I20260806 03:46:59.628553 15651 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.253.254:34329 every 8 connection(s)
I20260806 03:46:59.629132 15559 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785988004637995-12279-0/minicluster-data/master-0/data/info.pb
I20260806 03:46:59.629936 15653 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:46:59.632341 15653 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5c7a452da8e24f42a31e623986fcef71: Bootstrap starting.
I20260806 03:46:59.632941 15653 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5c7a452da8e24f42a31e623986fcef71: Neither blocks nor log segments found. Creating new log.
I20260806 03:46:59.633252 15653 log.cc:824] T 00000000000000000000000000000000 P 5c7a452da8e24f42a31e623986fcef71: Log is configured to *not* fsync() on all Append() calls
I20260806 03:46:59.634008 15653 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5c7a452da8e24f42a31e623986fcef71: No bootstrap required, opened a new log
I20260806 03:46:59.635388 12279 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskToVWRe/build/release/bin/kudu as pid 15559
I20260806 03:46:59.635490 12279 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785988004637995-12279-0/minicluster-data/master-0/wal/instance
I20260806 03:46:59.635672 12279 external_mini_cluster.cc:1428] Setting key 664146918103ac263fc6a2d0d7b0f624
I20260806 03:46:59.635978 15653 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5c7a452da8e24f42a31e623986fcef71 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c7a452da8e24f42a31e623986fcef71" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 34329 } }
I20260806 03:46:59.636202 15653 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5c7a452da8e24f42a31e623986fcef71 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:46:59.636293 15653 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5c7a452da8e24f42a31e623986fcef71 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c7a452da8e24f42a31e623986fcef71, State: Initialized, Role: FOLLOWER
I20260806 03:46:59.636474 15653 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5c7a452da8e24f42a31e623986fcef71 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c7a452da8e24f42a31e623986fcef71" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 34329 } }
I20260806 03:46:59.636586 15653 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5c7a452da8e24f42a31e623986fcef71 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 03:46:59.636631 15653 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5c7a452da8e24f42a31e623986fcef71 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 03:46:59.636682 15653 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5c7a452da8e24f42a31e623986fcef71 [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:46:59.637449 15653 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5c7a452da8e24f42a31e623986fcef71 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c7a452da8e24f42a31e623986fcef71" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 34329 } }
I20260806 03:46:59.637569 15653 leader_election.cc:302] T 00000000000000000000000000000000 P 5c7a452da8e24f42a31e623986fcef71 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5c7a452da8e24f42a31e623986fcef71; no voters:
I20260806 03:46:59.637746 15653 leader_election.cc:288] T 00000000000000000000000000000000 P 5c7a452da8e24f42a31e623986fcef71 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 03:46:59.637775 15660 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5c7a452da8e24f42a31e623986fcef71 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 03:46:59.637915 15660 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5c7a452da8e24f42a31e623986fcef71 [term 1 LEADER]: Becoming Leader. State: Replica: 5c7a452da8e24f42a31e623986fcef71, State: Running, Role: LEADER
I20260806 03:46:59.638054 15653 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5c7a452da8e24f42a31e623986fcef71 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 03:46:59.638065 15660 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5c7a452da8e24f42a31e623986fcef71 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c7a452da8e24f42a31e623986fcef71" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 34329 } }
I20260806 03:46:59.638604 15664 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5c7a452da8e24f42a31e623986fcef71 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5c7a452da8e24f42a31e623986fcef71" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c7a452da8e24f42a31e623986fcef71" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 34329 } } }
I20260806 03:46:59.638626 15665 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5c7a452da8e24f42a31e623986fcef71 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5c7a452da8e24f42a31e623986fcef71. Latest consensus state: current_term: 1 leader_uuid: "5c7a452da8e24f42a31e623986fcef71" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c7a452da8e24f42a31e623986fcef71" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 34329 } } }
I20260806 03:46:59.638808 15664 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5c7a452da8e24f42a31e623986fcef71 [sys.catalog]: This master's current role is: LEADER
I20260806 03:46:59.638816 15665 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5c7a452da8e24f42a31e623986fcef71 [sys.catalog]: This master's current role is: LEADER
I20260806 03:46:59.639329 15676 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 03:46:59.639932 15676 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 03:46:59.641984 15676 catalog_manager.cc:1347] Generated new cluster ID: 9646a07d94f8484d929029b4b5d224c0
I20260806 03:46:59.642042 15676 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 03:46:59 dist-test-slave-6ntk krb5kdc[15543](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988019, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.253.254@KRBTEST.COM
I20260806 03:46:59.647162 15659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:46:59.636219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38428 (local address 127.11.253.254:34329)
0806 03:46:59.636415 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0806 03:46:59.636424 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 03:46:59.636461 (+ 37us) server_negotiation.cc:407] Connection header received
0806 03:46:59.636553 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:46:59.636559 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:46:59.636622 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:46:59.636789 (+ 167us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:46:59.637673 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:46:59.638951 (+ 1278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:46:59.640066 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:46:59.640671 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:46:59.640781 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:46:59.643259 (+ 2478us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:46:59.643263 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:46:59.643264 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:46:59.643275 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:46:59.645034 (+ 1759us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:46:59.645420 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:46:59.645423 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:46:59.645423 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:46:59.646072 (+ 649us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:46:59.646297 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:46:59.646298 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:46:59.646299 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:46:59.646539 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:46:59.646720 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0806 03:46:59.646821 (+ 101us) server_negotiation.cc:299] Negotiation successful
0806 03:46:59.646980 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":91,"threads_started":1}
I20260806 03:46:59.651717 15676 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 03:46:59.652456 15676 catalog_manager.cc:1504] Loading token signing keys...
I20260806 03:46:59.659569 15676 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5c7a452da8e24f42a31e623986fcef71: Generated new TSK 0
I20260806 03:46:59.659755 15676 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.253.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.253.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.253.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.193.keytab.
Entry for principal kudu/127.11.253.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.193.keytab.
I20260806 03:46:59.689541 12279 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskToVWRe/build/release/bin/kudu
/tmp/dist-test-taskToVWRe/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785988004637995-12279-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785988004637995-12279-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785988004637995-12279-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785988004637995-12279-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.253.193:0
--local_ip_for_outbound_sockets=127.11.253.193
--webserver_interface=127.11.253.193
--webserver_port=0
--tserver_master_addrs=127.11.253.254:34329
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.193.keytab
--principal=kudu/127.11.253.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.253.212:45223
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785988004637995-12279-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785988004637995-12279-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785988004637995-12279-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:46:59.774639 15688 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:46:59.774873 15688 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:46:59.774933 15688 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:46:59.776124 15688 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:46:59.776196 15688 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:46:59.776242 15688 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.253.193
I20260806 03:46:59.777722 15688 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.253.212:45223
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785988004637995-12279-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785988004637995-12279-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.193.keytab
--principal=kudu/127.11.253.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.253.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785988004637995-12279-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.253.193
--webserver_port=0
--tserver_master_addrs=127.11.253.254:34329
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15688
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.253.193
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785988004637995-12279-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:46:59.778021 15688 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:46:59.778261 15688 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:46:59.783515 15694 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:46:59.783622 15695 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:46:59.783630 15688 server_base.cc:1034] running on GCE node
W20260806 03:46:59.783548 15697 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:46:59.784068 15688 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:46:59.784348 15688 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:46:59.785539 15688 hybrid_clock.cc:648] HybridClock initialized: now 1785988019785515 us; error 43 us; skew 500 ppm
Aug 06 03:46:59 dist-test-slave-6ntk krb5kdc[15543](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988019, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.253.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:46:59.788437 15688 init.cc:377] Logged in from keytab as kudu/127.11.253.193@KRBTEST.COM (short username kudu)
I20260806 03:46:59.788904 15688 webserver.cc:460] Webserver started at http://127.11.253.193:45691/ using document root <none> and password file <none>
I20260806 03:46:59.789175 15688 fs_manager.cc:359] Metadata directory not provided
I20260806 03:46:59.789256 15688 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:46:59.789418 15688 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:46:59.790570 15688 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785988004637995-12279-0/minicluster-data/ts-0/data/instance:
uuid: "c350fb195f0d4d07b0e03f83ec206161"
format_stamp: "Formatted at 2026-08-06 03:46:59 on dist-test-slave-6ntk"
server_key: "20c563c11c02b8f7afa3a14566d9baf6"
server_key_iv: "19df611bc64d7157d309ee3406765067"
server_key_version: "encryptionkey@0"
I20260806 03:46:59.790933 15688 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785988004637995-12279-0/minicluster-data/ts-0/wal/instance:
uuid: "c350fb195f0d4d07b0e03f83ec206161"
format_stamp: "Formatted at 2026-08-06 03:46:59 on dist-test-slave-6ntk"
server_key: "20c563c11c02b8f7afa3a14566d9baf6"
server_key_iv: "19df611bc64d7157d309ee3406765067"
server_key_version: "encryptionkey@0"
I20260806 03:46:59.792315 15688 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 03:46:59.793094 15706 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:46:59.793347 15688 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 03:46:59.793460 15688 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785988004637995-12279-0/minicluster-data/ts-0/data,/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785988004637995-12279-0/minicluster-data/ts-0/wal
uuid: "c350fb195f0d4d07b0e03f83ec206161"
format_stamp: "Formatted at 2026-08-06 03:46:59 on dist-test-slave-6ntk"
server_key: "20c563c11c02b8f7afa3a14566d9baf6"
server_key_iv: "19df611bc64d7157d309ee3406765067"
server_key_version: "encryptionkey@0"
I20260806 03:46:59.793537 15688 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785988004637995-12279-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785988004637995-12279-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785988004637995-12279-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:46:59.843171 15688 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:46:59.845136 15688 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:46:59.845314 15688 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:46:59.845563 15688 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 03:46:59.845916 15688 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 03:46:59.845988 15688 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 03:46:59.846035 15688 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 03:46:59.846063 15688 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 03:46:59.851894 15688 rpc_server.cc:307] RPC server started. Bound to: 127.11.253.193:42143
I20260806 03:46:59.851958 15881 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.253.193:42143 every 8 connection(s)
I20260806 03:46:59.852294 15688 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785988004637995-12279-0/minicluster-data/ts-0/data/info.pb
I20260806 03:46:59.855935 12279 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskToVWRe/build/release/bin/kudu as pid 15688
I20260806 03:46:59.856040 12279 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785988004637995-12279-0/minicluster-data/ts-0/wal/instance
I20260806 03:46:59.856230 12279 external_mini_cluster.cc:1428] Setting key 0aef49eb362892dd85898b6f4cf390dc
Aug 06 03:46:59 dist-test-slave-6ntk krb5kdc[15543](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988019, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.253.193@KRBTEST.COM for kudu/127.11.253.254@KRBTEST.COM
I20260806 03:46:59.862625 15659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:46:59.852988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.253.193:58259 (local address 127.11.253.254:34329)
0806 03:46:59.853058 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0806 03:46:59.853063 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 03:46:59.853292 (+ 229us) server_negotiation.cc:407] Connection header received
0806 03:46:59.853810 (+ 518us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:46:59.853811 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:46:59.853826 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:46:59.853953 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:46:59.854915 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:46:59.855400 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:46:59.856513 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:46:59.856747 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:46:59.856826 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:46:59.859454 (+ 2628us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:46:59.859455 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:46:59.859456 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:46:59.859464 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:46:59.860971 (+ 1507us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:46:59.861494 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:46:59.861495 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:46:59.861496 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:46:59.861553 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:46:59.861848 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:46:59.861849 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:46:59.861849 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:46:59.862036 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:46:59.862094 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0806 03:46:59.862476 (+ 382us) server_negotiation.cc:299] Negotiation successful
0806 03:46:59.862556 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260806 03:46:59.862911 15886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:46:59.852960 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:34329 (local address 127.11.253.193:58259)
0806 03:46:59.853135 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0806 03:46:59.853178 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 03:46:59.853694 (+ 516us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:46:59.854040 (+ 346us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:46:59.854043 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:46:59.854611 (+ 568us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:46:59.854829 (+ 218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:46:59.854834 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:46:59.855479 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:46:59.855480 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:46:59.856415 (+ 935us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:46:59.856418 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:46:59.856934 (+ 516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:46:59.856936 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:46:59.857050 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:46:59.857546 (+ 496us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:46:59.857554 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:46:59.859347 (+ 1793us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:46:59.861065 (+ 1718us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:46:59.861073 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:46:59.861075 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:46:59.861407 (+ 332us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:46:59.861624 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:46:59.861625 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:46:59.861626 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:46:59.861786 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:46:59.862111 (+ 325us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:46:59.862120 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:46:59.862292 (+ 172us) client_negotiation.cc:769] Sending connection context
0806 03:46:59.862431 (+ 139us) client_negotiation.cc:240] Negotiation successful
0806 03:46:59.862645 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":88,"threads_started":1}
I20260806 03:46:59.863497 15882 heartbeater.cc:347] Connected to a master server at 127.11.253.254:34329
I20260806 03:46:59.863610 15882 heartbeater.cc:464] Registering TS with master...
I20260806 03:46:59.863835 15882 heartbeater.cc:511] Master 127.11.253.254:34329 requested a full tablet report, sending...
I20260806 03:46:59.864467 15598 ts_manager.cc:194] Registered new tserver with Master: c350fb195f0d4d07b0e03f83ec206161 (127.11.253.193:42143)
I20260806 03:46:59.865154 15598 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.253.193@KRBTEST.COM'} at 127.11.253.193:58259
WARNING: no policy specified for kudu/127.11.253.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.253.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.253.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.194.keytab.
Entry for principal kudu/127.11.253.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.194.keytab.
I20260806 03:46:59.885739 12279 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskToVWRe/build/release/bin/kudu
/tmp/dist-test-taskToVWRe/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785988004637995-12279-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785988004637995-12279-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785988004637995-12279-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785988004637995-12279-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.253.194:0
--local_ip_for_outbound_sockets=127.11.253.194
--webserver_interface=127.11.253.194
--webserver_port=0
--tserver_master_addrs=127.11.253.254:34329
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.194.keytab
--principal=kudu/127.11.253.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.253.212:45223
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785988004637995-12279-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785988004637995-12279-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785988004637995-12279-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:46:59.973392 15890 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:46:59.973639 15890 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:46:59.973681 15890 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:46:59.974915 15890 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:46:59.974990 15890 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:46:59.975037 15890 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.253.194
I20260806 03:46:59.976503 15890 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.253.212:45223
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785988004637995-12279-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785988004637995-12279-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.194.keytab
--principal=kudu/127.11.253.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.253.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785988004637995-12279-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.253.194
--webserver_port=0
--tserver_master_addrs=127.11.253.254:34329
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15890
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.253.194
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785988004637995-12279-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:46:59.976789 15890 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:46:59.977028 15890 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:46:59.982331 15897 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:46:59.982338 15900 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:46:59.982297 15898 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:46:59.982522 15890 server_base.cc:1034] running on GCE node
I20260806 03:46:59.982724 15890 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:46:59.983007 15890 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:46:59.984162 15890 hybrid_clock.cc:648] HybridClock initialized: now 1785988019984132 us; error 37 us; skew 500 ppm
Aug 06 03:46:59 dist-test-slave-6ntk krb5kdc[15543](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988019, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.253.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:46:59.986603 15890 init.cc:377] Logged in from keytab as kudu/127.11.253.194@KRBTEST.COM (short username kudu)
I20260806 03:46:59.987057 15890 webserver.cc:460] Webserver started at http://127.11.253.194:41169/ using document root <none> and password file <none>
I20260806 03:46:59.987299 15890 fs_manager.cc:359] Metadata directory not provided
I20260806 03:46:59.987368 15890 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:46:59.987524 15890 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:46:59.988593 15890 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785988004637995-12279-0/minicluster-data/ts-1/data/instance:
uuid: "47f75597049f4e099a5bc7a45b7cbfa4"
format_stamp: "Formatted at 2026-08-06 03:46:59 on dist-test-slave-6ntk"
server_key: "276c85f073a465236ad4e6ea8176ba32"
server_key_iv: "d57248b7c0f067fedadac0734be4ba1b"
server_key_version: "encryptionkey@0"
I20260806 03:46:59.988976 15890 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785988004637995-12279-0/minicluster-data/ts-1/wal/instance:
uuid: "47f75597049f4e099a5bc7a45b7cbfa4"
format_stamp: "Formatted at 2026-08-06 03:46:59 on dist-test-slave-6ntk"
server_key: "276c85f073a465236ad4e6ea8176ba32"
server_key_iv: "d57248b7c0f067fedadac0734be4ba1b"
server_key_version: "encryptionkey@0"
I20260806 03:46:59.990419 15890 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 03:46:59.991155 15909 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:46:59.991338 15890 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 03:46:59.991447 15890 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785988004637995-12279-0/minicluster-data/ts-1/data,/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785988004637995-12279-0/minicluster-data/ts-1/wal
uuid: "47f75597049f4e099a5bc7a45b7cbfa4"
format_stamp: "Formatted at 2026-08-06 03:46:59 on dist-test-slave-6ntk"
server_key: "276c85f073a465236ad4e6ea8176ba32"
server_key_iv: "d57248b7c0f067fedadac0734be4ba1b"
server_key_version: "encryptionkey@0"
I20260806 03:46:59.991536 15890 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785988004637995-12279-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785988004637995-12279-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785988004637995-12279-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:47:00.012079 15890 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:47:00.014176 15890 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:47:00.014351 15890 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:47:00.014627 15890 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 03:47:00.014959 15890 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 03:47:00.015017 15890 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 03:47:00.015062 15890 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 03:47:00.015090 15890 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 03:47:00.020928 15890 rpc_server.cc:307] RPC server started. Bound to: 127.11.253.194:37811
I20260806 03:47:00.021013 16069 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.253.194:37811 every 8 connection(s)
I20260806 03:47:00.021368 15890 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785988004637995-12279-0/minicluster-data/ts-1/data/info.pb
I20260806 03:47:00.024811 12279 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskToVWRe/build/release/bin/kudu as pid 15890
I20260806 03:47:00.024940 12279 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785988004637995-12279-0/minicluster-data/ts-1/wal/instance
I20260806 03:47:00.025116 12279 external_mini_cluster.cc:1428] Setting key 0d46afda598e4f0940feccc0ab5c9018
Aug 06 03:47:00 dist-test-slave-6ntk krb5kdc[15543](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988019, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.253.194@KRBTEST.COM for kudu/127.11.253.254@KRBTEST.COM
I20260806 03:47:00.031709 15659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:00.022180 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.253.194:40191 (local address 127.11.253.254:34329)
0806 03:47:00.022254 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0806 03:47:00.022257 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 03:47:00.022490 (+ 233us) server_negotiation.cc:407] Connection header received
0806 03:47:00.023045 (+ 555us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:47:00.023047 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:47:00.023060 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:47:00.023107 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:47:00.024112 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:00.024632 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:00.025935 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:00.026153 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:00.026242 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:00.028603 (+ 2361us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:47:00.028605 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:47:00.028605 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:47:00.028614 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:47:00.030261 (+ 1647us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:47:00.030685 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:47:00.030686 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:47:00.030687 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:47:00.030748 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:47:00.031023 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:47:00.031024 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:47:00.031025 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:47:00.031153 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:47:00.031227 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0806 03:47:00.031514 (+ 287us) server_negotiation.cc:299] Negotiation successful
0806 03:47:00.031616 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260806 03:47:00.031916 16072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:00.022151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:34329 (local address 127.11.253.194:40191)
0806 03:47:00.022356 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0806 03:47:00.022394 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0806 03:47:00.022933 (+ 539us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:47:00.023176 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:47:00.023180 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:47:00.023789 (+ 609us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:47:00.023994 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:00.023998 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:00.024732 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:00.024733 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:00.025815 (+ 1082us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:00.025817 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:00.026254 (+ 437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:00.026255 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:00.026405 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:00.026977 (+ 572us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:47:00.026985 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:47:00.028496 (+ 1511us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:47:00.030362 (+ 1866us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:47:00.030378 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:47:00.030380 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:47:00.030599 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:47:00.030827 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:47:00.030828 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:47:00.030828 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:47:00.030946 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:47:00.031229 (+ 283us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:47:00.031237 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:47:00.031376 (+ 139us) client_negotiation.cc:769] Sending connection context
0806 03:47:00.031481 (+ 105us) client_negotiation.cc:240] Negotiation successful
0806 03:47:00.031648 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":99,"threads_started":1}
I20260806 03:47:00.032384 16070 heartbeater.cc:347] Connected to a master server at 127.11.253.254:34329
I20260806 03:47:00.032493 16070 heartbeater.cc:464] Registering TS with master...
I20260806 03:47:00.032711 16070 heartbeater.cc:511] Master 127.11.253.254:34329 requested a full tablet report, sending...
I20260806 03:47:00.033111 15598 ts_manager.cc:194] Registered new tserver with Master: 47f75597049f4e099a5bc7a45b7cbfa4 (127.11.253.194:37811)
I20260806 03:47:00.033507 15598 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.253.194@KRBTEST.COM'} at 127.11.253.194:40191
WARNING: no policy specified for kudu/127.11.253.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.253.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.253.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.195.keytab.
Entry for principal kudu/127.11.253.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.195.keytab.
I20260806 03:47:00.053273 12279 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskToVWRe/build/release/bin/kudu
/tmp/dist-test-taskToVWRe/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785988004637995-12279-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785988004637995-12279-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785988004637995-12279-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785988004637995-12279-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.253.195:0
--local_ip_for_outbound_sockets=127.11.253.195
--webserver_interface=127.11.253.195
--webserver_port=0
--tserver_master_addrs=127.11.253.254:34329
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.195.keytab
--principal=kudu/127.11.253.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.253.212:45223
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785988004637995-12279-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785988004637995-12279-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785988004637995-12279-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:47:00.138469 16076 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:47:00.138756 16076 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:47:00.138796 16076 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:47:00.140173 16076 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:47:00.140241 16076 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:47:00.140302 16076 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.253.195
I20260806 03:47:00.141777 16076 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.253.212:45223
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785988004637995-12279-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785988004637995-12279-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.195.keytab
--principal=kudu/127.11.253.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.253.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785988004637995-12279-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.253.195
--webserver_port=0
--tserver_master_addrs=127.11.253.254:34329
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16076
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.253.195
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785988004637995-12279-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:47:00.142060 16076 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:47:00.142300 16076 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:47:00.147404 16085 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:47:00.147404 16082 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:47:00.147639 16076 server_base.cc:1034] running on GCE node
W20260806 03:47:00.147420 16087 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:47:00.147971 16076 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:47:00.148229 16076 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:47:00.149398 16076 hybrid_clock.cc:648] HybridClock initialized: now 1785988020149384 us; error 33 us; skew 500 ppm
Aug 06 03:47:00 dist-test-slave-6ntk krb5kdc[15543](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988020, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.253.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:47:00.152124 16076 init.cc:377] Logged in from keytab as kudu/127.11.253.195@KRBTEST.COM (short username kudu)
I20260806 03:47:00.152496 16076 webserver.cc:460] Webserver started at http://127.11.253.195:34105/ using document root <none> and password file <none>
I20260806 03:47:00.152722 16076 fs_manager.cc:359] Metadata directory not provided
I20260806 03:47:00.152768 16076 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:47:00.152926 16076 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:47:00.154038 16076 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785988004637995-12279-0/minicluster-data/ts-2/data/instance:
uuid: "2f60522bd09c41b1b1eed7080b2ebaa2"
format_stamp: "Formatted at 2026-08-06 03:47:00 on dist-test-slave-6ntk"
server_key: "5702a519c06bee4cae79ead41f15e7ce"
server_key_iv: "a7e4be275e948506c5a2e95217b2caa6"
server_key_version: "encryptionkey@0"
I20260806 03:47:00.154397 16076 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785988004637995-12279-0/minicluster-data/ts-2/wal/instance:
uuid: "2f60522bd09c41b1b1eed7080b2ebaa2"
format_stamp: "Formatted at 2026-08-06 03:47:00 on dist-test-slave-6ntk"
server_key: "5702a519c06bee4cae79ead41f15e7ce"
server_key_iv: "a7e4be275e948506c5a2e95217b2caa6"
server_key_version: "encryptionkey@0"
I20260806 03:47:00.155709 16076 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260806 03:47:00.156422 16096 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:47:00.156599 16076 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 03:47:00.156692 16076 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785988004637995-12279-0/minicluster-data/ts-2/data,/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785988004637995-12279-0/minicluster-data/ts-2/wal
uuid: "2f60522bd09c41b1b1eed7080b2ebaa2"
format_stamp: "Formatted at 2026-08-06 03:47:00 on dist-test-slave-6ntk"
server_key: "5702a519c06bee4cae79ead41f15e7ce"
server_key_iv: "a7e4be275e948506c5a2e95217b2caa6"
server_key_version: "encryptionkey@0"
I20260806 03:47:00.156766 16076 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785988004637995-12279-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785988004637995-12279-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785988004637995-12279-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:47:00.164073 16076 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:47:00.166442 16076 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:47:00.166605 16076 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:47:00.166822 16076 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 03:47:00.167121 16076 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 03:47:00.167160 16076 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 03:47:00.167179 16076 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 03:47:00.167230 16076 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 03:47:00.173087 16076 rpc_server.cc:307] RPC server started. Bound to: 127.11.253.195:38477
I20260806 03:47:00.173132 16275 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.253.195:38477 every 8 connection(s)
I20260806 03:47:00.173439 16076 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785988004637995-12279-0/minicluster-data/ts-2/data/info.pb
I20260806 03:47:00.178834 12279 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskToVWRe/build/release/bin/kudu as pid 16076
I20260806 03:47:00.178947 12279 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785988004637995-12279-0/minicluster-data/ts-2/wal/instance
I20260806 03:47:00.179134 12279 external_mini_cluster.cc:1428] Setting key 7d288f33ea41c4668453c0fe353fcde4
Aug 06 03:47:00 dist-test-slave-6ntk krb5kdc[15543](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988020, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.253.195@KRBTEST.COM for kudu/127.11.253.254@KRBTEST.COM
I20260806 03:47:00.182796 15659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:00.174160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.253.195:47295 (local address 127.11.253.254:34329)
0806 03:47:00.174241 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0806 03:47:00.174243 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:47:00.174412 (+ 169us) server_negotiation.cc:407] Connection header received
0806 03:47:00.174872 (+ 460us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:47:00.174874 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:47:00.174884 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:47:00.174932 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:47:00.175816 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:00.176305 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:00.177248 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:00.177427 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:00.177478 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:00.179930 (+ 2452us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:47:00.179932 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:47:00.179932 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:47:00.179941 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:47:00.181386 (+ 1445us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:47:00.181796 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:47:00.181797 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:47:00.181797 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:47:00.181858 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:47:00.182212 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:47:00.182213 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:47:00.182213 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:47:00.182334 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:47:00.182393 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0806 03:47:00.182649 (+ 256us) server_negotiation.cc:299] Negotiation successful
0806 03:47:00.182716 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260806 03:47:00.182991 16280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:00.174119 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:34329 (local address 127.11.253.195:47295)
0806 03:47:00.174291 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0806 03:47:00.174328 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0806 03:47:00.174779 (+ 451us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:47:00.174989 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:47:00.174992 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:47:00.175543 (+ 551us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:47:00.175724 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:00.175728 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:00.176379 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:00.176379 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:00.177174 (+ 795us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:00.177176 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:00.177498 (+ 322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:00.177498 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:00.177599 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:00.178136 (+ 537us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:47:00.178144 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:47:00.179835 (+ 1691us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:47:00.181458 (+ 1623us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:47:00.181464 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:47:00.181466 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:47:00.181670 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:47:00.181996 (+ 326us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:47:00.181997 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:47:00.181997 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:47:00.182133 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:47:00.182404 (+ 271us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:47:00.182410 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:47:00.182527 (+ 117us) client_negotiation.cc:769] Sending connection context
0806 03:47:00.182621 (+ 94us) client_negotiation.cc:240] Negotiation successful
0806 03:47:00.182757 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":103,"threads_started":1}
I20260806 03:47:00.183547 16278 heartbeater.cc:347] Connected to a master server at 127.11.253.254:34329
I20260806 03:47:00.183656 16278 heartbeater.cc:464] Registering TS with master...
I20260806 03:47:00.183851 16278 heartbeater.cc:511] Master 127.11.253.254:34329 requested a full tablet report, sending...
I20260806 03:47:00.184273 15598 ts_manager.cc:194] Registered new tserver with Master: 2f60522bd09c41b1b1eed7080b2ebaa2 (127.11.253.195:38477)
I20260806 03:47:00.184618 15598 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.253.195@KRBTEST.COM'} at 127.11.253.195:47295
I20260806 03:47:00.190855 12279 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260806 03:47:00.205996 15659 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 03:47:00.204992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38442 (local address 127.11.253.254:34329)
0806 03:47:00.205100 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0806 03:47:00.205102 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:47:00.205161 (+ 59us) server_negotiation.cc:407] Connection header received
0806 03:47:00.205227 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:47:00.205228 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:47:00.205240 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:47:00.205284 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:47:00.205853 (+ 569us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38442: BlockingRecv error: recv got EOF from 127.0.0.1:38442 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
I20260806 03:47:00.206897 12279 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskToVWRe/build/release/bin/kudu with pid 15688
I20260806 03:47:00.223325 12279 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskToVWRe/build/release/bin/kudu with pid 15890
I20260806 03:47:00.233274 12279 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskToVWRe/build/release/bin/kudu with pid 16076
I20260806 03:47:00.242579 12279 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskToVWRe/build/release/bin/kudu with pid 15559
2026-08-06T03:47:00Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (2086 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785988004637995-12279-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 03:47:00 dist-test-slave-6ntk krb5kdc[16296](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 03:47:00 dist-test-slave-6ntk krb5kdc[16296](info): set up 2 sockets
Aug 06 03:47:00 dist-test-slave-6ntk krb5kdc[16296](info): commencing operation
krb5kdc: starting...
W20260806 03:47:01.472146 12279 mini_kdc.cc:121] Time spent starting KDC: real 1.205s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785988004637995-12279-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785988004637995-12279-0/krb5kdc/test-user.keytab.
Aug 06 03:47:01 dist-test-slave-6ntk krb5kdc[16296](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988021, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T03:47:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T03:47:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.253.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.253.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.253.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.254.keytab.
Entry for principal kudu/127.11.253.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.254.keytab.
I20260806 03:47:01.602070 12279 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskToVWRe/build/release/bin/kudu
/tmp/dist-test-taskToVWRe/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785988004637995-12279-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785988004637995-12279-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785988004637995-12279-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785988004637995-12279-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.253.254:45649
--webserver_interface=127.11.253.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.254.keytab
--principal=kudu/127.11.253.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.253.212:45085
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.253.254:45649
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785988004637995-12279-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785988004637995-12279-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785988004637995-12279-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:47:01.687129 16317 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:47:01.687345 16317 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:47:01.687367 16317 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:47:01.688534 16317 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 03:47:01.688576 16317 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:47:01.688591 16317 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:47:01.688603 16317 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 03:47:01.688616 16317 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 03:47:01.690169 16317 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.253.212:45085
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785988004637995-12279-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785988004637995-12279-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.253.254:45649
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.254.keytab
--principal=kudu/127.11.253.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.253.254:45649
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785988004637995-12279-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.253.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16317
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785988004637995-12279-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:47:01.690425 16317 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:47:01.690663 16317 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:47:01.695911 16325 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:47:01.695919 16323 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:47:01.696310 16328 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:47:01.696689 16317 server_base.cc:1034] running on GCE node
I20260806 03:47:01.697036 16317 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:47:01.697433 16317 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:47:01.698596 16317 hybrid_clock.cc:648] HybridClock initialized: now 1785988021698582 us; error 28 us; skew 500 ppm
Aug 06 03:47:01 dist-test-slave-6ntk krb5kdc[16296](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988021, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.253.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:47:01.707291 16317 init.cc:377] Logged in from keytab as kudu/127.11.253.254@KRBTEST.COM (short username kudu)
I20260806 03:47:01.707803 16317 webserver.cc:460] Webserver started at http://127.11.253.254:40233/ using document root <none> and password file <none>
I20260806 03:47:01.708029 16317 fs_manager.cc:359] Metadata directory not provided
I20260806 03:47:01.708071 16317 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:47:01.708161 16317 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:47:01.709213 16317 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785988004637995-12279-0/minicluster-data/master-0/data/instance:
uuid: "64f26e7bcf2c482bb45b0971f2d08737"
format_stamp: "Formatted at 2026-08-06 03:47:01 on dist-test-slave-6ntk"
server_key: "a2d613022a292a0a74d305353751844f"
server_key_iv: "7ff11c0818e0a7fd64ac7ca1797f7cb5"
server_key_version: "encryptionkey@0"
I20260806 03:47:01.709519 16317 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785988004637995-12279-0/minicluster-data/master-0/wal/instance:
uuid: "64f26e7bcf2c482bb45b0971f2d08737"
format_stamp: "Formatted at 2026-08-06 03:47:01 on dist-test-slave-6ntk"
server_key: "a2d613022a292a0a74d305353751844f"
server_key_iv: "7ff11c0818e0a7fd64ac7ca1797f7cb5"
server_key_version: "encryptionkey@0"
I20260806 03:47:01.710901 16317 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 03:47:01.711593 16337 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:47:01.711771 16317 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 03:47:01.711874 16317 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785988004637995-12279-0/minicluster-data/master-0/data,/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785988004637995-12279-0/minicluster-data/master-0/wal
uuid: "64f26e7bcf2c482bb45b0971f2d08737"
format_stamp: "Formatted at 2026-08-06 03:47:01 on dist-test-slave-6ntk"
server_key: "a2d613022a292a0a74d305353751844f"
server_key_iv: "7ff11c0818e0a7fd64ac7ca1797f7cb5"
server_key_version: "encryptionkey@0"
I20260806 03:47:01.711958 16317 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785988004637995-12279-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785988004637995-12279-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785988004637995-12279-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:47:01.725637 16317 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:47:01.727769 16317 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:47:01.727943 16317 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:47:01.732280 16317 rpc_server.cc:307] RPC server started. Bound to: 127.11.253.254:45649
I20260806 03:47:01.732314 16409 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.253.254:45649 every 8 connection(s)
I20260806 03:47:01.732805 16317 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785988004637995-12279-0/minicluster-data/master-0/data/info.pb
I20260806 03:47:01.733484 16410 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:47:01.735961 16410 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 64f26e7bcf2c482bb45b0971f2d08737: Bootstrap starting.
I20260806 03:47:01.736512 16410 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 64f26e7bcf2c482bb45b0971f2d08737: Neither blocks nor log segments found. Creating new log.
I20260806 03:47:01.736773 16410 log.cc:824] T 00000000000000000000000000000000 P 64f26e7bcf2c482bb45b0971f2d08737: Log is configured to *not* fsync() on all Append() calls
I20260806 03:47:01.737421 16410 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 64f26e7bcf2c482bb45b0971f2d08737: No bootstrap required, opened a new log
I20260806 03:47:01.737967 12279 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskToVWRe/build/release/bin/kudu as pid 16317
I20260806 03:47:01.738059 12279 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785988004637995-12279-0/minicluster-data/master-0/wal/instance
I20260806 03:47:01.738257 12279 external_mini_cluster.cc:1428] Setting key 88fc3928000300205ef92f1f1d7bae65
I20260806 03:47:01.740079 16410 raft_consensus.cc:348] T 00000000000000000000000000000000 P 64f26e7bcf2c482bb45b0971f2d08737 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64f26e7bcf2c482bb45b0971f2d08737" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 45649 } }
I20260806 03:47:01.740294 16410 raft_consensus.cc:374] T 00000000000000000000000000000000 P 64f26e7bcf2c482bb45b0971f2d08737 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:47:01.740355 16410 raft_consensus.cc:729] T 00000000000000000000000000000000 P 64f26e7bcf2c482bb45b0971f2d08737 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64f26e7bcf2c482bb45b0971f2d08737, State: Initialized, Role: FOLLOWER
I20260806 03:47:01.740500 16410 consensus_queue.cc:260] T 00000000000000000000000000000000 P 64f26e7bcf2c482bb45b0971f2d08737 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64f26e7bcf2c482bb45b0971f2d08737" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 45649 } }
I20260806 03:47:01.740593 16410 raft_consensus.cc:388] T 00000000000000000000000000000000 P 64f26e7bcf2c482bb45b0971f2d08737 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 03:47:01.740628 16410 raft_consensus.cc:482] T 00000000000000000000000000000000 P 64f26e7bcf2c482bb45b0971f2d08737 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 03:47:01.740680 16410 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 64f26e7bcf2c482bb45b0971f2d08737 [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:47:01.741360 16410 raft_consensus.cc:504] T 00000000000000000000000000000000 P 64f26e7bcf2c482bb45b0971f2d08737 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64f26e7bcf2c482bb45b0971f2d08737" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 45649 } }
I20260806 03:47:01.741494 16410 leader_election.cc:302] T 00000000000000000000000000000000 P 64f26e7bcf2c482bb45b0971f2d08737 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 64f26e7bcf2c482bb45b0971f2d08737; no voters:
I20260806 03:47:01.741691 16410 leader_election.cc:288] T 00000000000000000000000000000000 P 64f26e7bcf2c482bb45b0971f2d08737 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 03:47:01.741727 16416 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 64f26e7bcf2c482bb45b0971f2d08737 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 03:47:01.741845 16416 raft_consensus.cc:686] T 00000000000000000000000000000000 P 64f26e7bcf2c482bb45b0971f2d08737 [term 1 LEADER]: Becoming Leader. State: Replica: 64f26e7bcf2c482bb45b0971f2d08737, State: Running, Role: LEADER
I20260806 03:47:01.742012 16410 sys_catalog.cc:564] T 00000000000000000000000000000000 P 64f26e7bcf2c482bb45b0971f2d08737 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 03:47:01.741998 16416 consensus_queue.cc:237] T 00000000000000000000000000000000 P 64f26e7bcf2c482bb45b0971f2d08737 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64f26e7bcf2c482bb45b0971f2d08737" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 45649 } }
I20260806 03:47:01.742543 16419 sys_catalog.cc:455] T 00000000000000000000000000000000 P 64f26e7bcf2c482bb45b0971f2d08737 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "64f26e7bcf2c482bb45b0971f2d08737" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64f26e7bcf2c482bb45b0971f2d08737" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 45649 } } }
I20260806 03:47:01.742834 16419 sys_catalog.cc:458] T 00000000000000000000000000000000 P 64f26e7bcf2c482bb45b0971f2d08737 [sys.catalog]: This master's current role is: LEADER
I20260806 03:47:01.742740 16420 sys_catalog.cc:455] T 00000000000000000000000000000000 P 64f26e7bcf2c482bb45b0971f2d08737 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 64f26e7bcf2c482bb45b0971f2d08737. Latest consensus state: current_term: 1 leader_uuid: "64f26e7bcf2c482bb45b0971f2d08737" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64f26e7bcf2c482bb45b0971f2d08737" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 45649 } } }
I20260806 03:47:01.743309 16430 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 03:47:01.743958 16430 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 03:47:01.743335 16420 sys_catalog.cc:458] T 00000000000000000000000000000000 P 64f26e7bcf2c482bb45b0971f2d08737 [sys.catalog]: This master's current role is: LEADER
I20260806 03:47:01.745656 16430 catalog_manager.cc:1347] Generated new cluster ID: 3679b23833ff44ea8b5a6353dd7d6bfe
I20260806 03:47:01.745713 16430 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 03:47:01 dist-test-slave-6ntk krb5kdc[16296](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988021, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.253.254@KRBTEST.COM
I20260806 03:47:01.751653 16414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:01.738821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39360 (local address 127.11.253.254:45649)
0806 03:47:01.739000 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0806 03:47:01.739007 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 03:47:01.739032 (+ 25us) server_negotiation.cc:407] Connection header received
0806 03:47:01.740005 (+ 973us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:47:01.740013 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:47:01.740062 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:47:01.740176 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:47:01.741102 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:01.742318 (+ 1216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:01.743388 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:01.744007 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:01.744113 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:01.747247 (+ 3134us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:47:01.747251 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:47:01.747251 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:47:01.747262 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:47:01.749691 (+ 2429us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:47:01.750124 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:47:01.750127 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:47:01.750128 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:47:01.750229 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:47:01.750416 (+ 187us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:47:01.750417 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:47:01.750417 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:47:01.750689 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:47:01.750836 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0806 03:47:01.751286 (+ 450us) server_negotiation.cc:299] Negotiation successful
0806 03:47:01.751428 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":85,"threads_started":1}
I20260806 03:47:01.752523 12279 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 03:47:01.761917 16430 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 03:47:01.762742 16430 catalog_manager.cc:1504] Loading token signing keys...
I20260806 03:47:01.766733 16430 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 64f26e7bcf2c482bb45b0971f2d08737: Generated new TSK 0
I20260806 03:47:01.766948 16430 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 03:47:01.771395 16414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:01.767626 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39362 (local address 127.11.253.254:45649)
0806 03:47:01.767683 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0806 03:47:01.767686 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 03:47:01.767806 (+ 120us) server_negotiation.cc:407] Connection header received
0806 03:47:01.767843 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:47:01.767844 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:47:01.767855 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:47:01.767934 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:47:01.768281 (+ 347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:01.769076 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:01.769808 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:01.770144 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:01.770219 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:01.770922 (+ 703us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:47:01.770924 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:47:01.770924 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 03:47:01.771150 (+ 226us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:47:01.771169 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 03:47:01.771176 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:47:01.771234 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0806 03:47:01.771289 (+ 55us) server_negotiation.cc:299] Negotiation successful
0806 03:47:01.771318 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260806 03:47:01.773939 12279 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskToVWRe/build/release/bin/kudu with pid 16317
2026-08-06T03:47:01Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1538 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785988004637995-12279-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 03:47:01 dist-test-slave-6ntk krb5kdc[16455](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 03:47:01 dist-test-slave-6ntk krb5kdc[16455](info): set up 2 sockets
Aug 06 03:47:01 dist-test-slave-6ntk krb5kdc[16455](info): commencing operation
krb5kdc: starting...
W20260806 03:47:03.020221 12279 mini_kdc.cc:121] Time spent starting KDC: real 1.215s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785988004637995-12279-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785988004637995-12279-0/krb5kdc/test-user.keytab.
Aug 06 03:47:03 dist-test-slave-6ntk krb5kdc[16455](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988023, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T03:47:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T03:47:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.253.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.253.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.253.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.254.keytab.
Entry for principal kudu/127.11.253.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.254.keytab.
I20260806 03:47:03.153028 12279 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskToVWRe/build/release/bin/kudu
/tmp/dist-test-taskToVWRe/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785988004637995-12279-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785988004637995-12279-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785988004637995-12279-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785988004637995-12279-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.253.254:40845
--webserver_interface=127.11.253.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.254.keytab
--principal=kudu/127.11.253.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.253.212:43285
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.253.254:40845
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785988004637995-12279-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785988004637995-12279-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785988004637995-12279-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:47:03.239905 16472 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:47:03.240135 16472 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:47:03.240161 16472 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:47:03.241354 16472 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 03:47:03.241403 16472 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:47:03.241420 16472 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:47:03.241466 16472 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 03:47:03.241500 16472 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 03:47:03.243084 16472 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.253.212:43285
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785988004637995-12279-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785988004637995-12279-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.253.254:40845
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.254.keytab
--principal=kudu/127.11.253.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.253.254:40845
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785988004637995-12279-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.253.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16472
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785988004637995-12279-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:47:03.243376 16472 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:47:03.243630 16472 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:47:03.248955 16484 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:47:03.248951 16482 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:47:03.248965 16481 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:47:03.248976 16472 server_base.cc:1034] running on GCE node
I20260806 03:47:03.249464 16472 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:47:03.249717 16472 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:47:03.250864 16472 hybrid_clock.cc:648] HybridClock initialized: now 1785988023250851 us; error 35 us; skew 500 ppm
Aug 06 03:47:03 dist-test-slave-6ntk krb5kdc[16455](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988023, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.253.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:47:03.253378 16472 init.cc:377] Logged in from keytab as kudu/127.11.253.254@KRBTEST.COM (short username kudu)
I20260806 03:47:03.253813 16472 webserver.cc:460] Webserver started at http://127.11.253.254:34599/ using document root <none> and password file <none>
I20260806 03:47:03.254083 16472 fs_manager.cc:359] Metadata directory not provided
I20260806 03:47:03.254158 16472 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:47:03.254311 16472 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:47:03.255415 16472 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785988004637995-12279-0/minicluster-data/master-0/data/instance:
uuid: "466a4788b36d43aaa0688d36c1160682"
format_stamp: "Formatted at 2026-08-06 03:47:03 on dist-test-slave-6ntk"
server_key: "09e98b00484e1ce5550ca7d71c97d758"
server_key_iv: "605c2d8d05bd5375e2a38d404e599748"
server_key_version: "encryptionkey@0"
I20260806 03:47:03.255788 16472 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785988004637995-12279-0/minicluster-data/master-0/wal/instance:
uuid: "466a4788b36d43aaa0688d36c1160682"
format_stamp: "Formatted at 2026-08-06 03:47:03 on dist-test-slave-6ntk"
server_key: "09e98b00484e1ce5550ca7d71c97d758"
server_key_iv: "605c2d8d05bd5375e2a38d404e599748"
server_key_version: "encryptionkey@0"
I20260806 03:47:03.257164 16472 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 03:47:03.258025 16496 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:47:03.258221 16472 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 03:47:03.258328 16472 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785988004637995-12279-0/minicluster-data/master-0/data,/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785988004637995-12279-0/minicluster-data/master-0/wal
uuid: "466a4788b36d43aaa0688d36c1160682"
format_stamp: "Formatted at 2026-08-06 03:47:03 on dist-test-slave-6ntk"
server_key: "09e98b00484e1ce5550ca7d71c97d758"
server_key_iv: "605c2d8d05bd5375e2a38d404e599748"
server_key_version: "encryptionkey@0"
I20260806 03:47:03.258440 16472 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785988004637995-12279-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785988004637995-12279-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785988004637995-12279-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:47:03.279623 16472 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:47:03.281787 16472 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:47:03.282006 16472 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:47:03.286188 16472 rpc_server.cc:307] RPC server started. Bound to: 127.11.253.254:40845
I20260806 03:47:03.286389 16572 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.253.254:40845 every 8 connection(s)
I20260806 03:47:03.286674 16472 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785988004637995-12279-0/minicluster-data/master-0/data/info.pb
I20260806 03:47:03.287662 16573 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:47:03.289110 12279 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskToVWRe/build/release/bin/kudu as pid 16472
I20260806 03:47:03.289223 12279 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785988004637995-12279-0/minicluster-data/master-0/wal/instance
I20260806 03:47:03.289419 12279 external_mini_cluster.cc:1428] Setting key 23c3a12a626436cf7f268dfd36bdfd72
I20260806 03:47:03.290593 16573 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 466a4788b36d43aaa0688d36c1160682: Bootstrap starting.
I20260806 03:47:03.291389 16573 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 466a4788b36d43aaa0688d36c1160682: Neither blocks nor log segments found. Creating new log.
I20260806 03:47:03.291749 16573 log.cc:824] T 00000000000000000000000000000000 P 466a4788b36d43aaa0688d36c1160682: Log is configured to *not* fsync() on all Append() calls
I20260806 03:47:03.292490 16573 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 466a4788b36d43aaa0688d36c1160682: No bootstrap required, opened a new log
I20260806 03:47:03.295073 16573 raft_consensus.cc:348] T 00000000000000000000000000000000 P 466a4788b36d43aaa0688d36c1160682 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "466a4788b36d43aaa0688d36c1160682" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 40845 } }
I20260806 03:47:03.295274 16573 raft_consensus.cc:374] T 00000000000000000000000000000000 P 466a4788b36d43aaa0688d36c1160682 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:47:03.295320 16573 raft_consensus.cc:729] T 00000000000000000000000000000000 P 466a4788b36d43aaa0688d36c1160682 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 466a4788b36d43aaa0688d36c1160682, State: Initialized, Role: FOLLOWER
I20260806 03:47:03.295498 16573 consensus_queue.cc:260] T 00000000000000000000000000000000 P 466a4788b36d43aaa0688d36c1160682 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "466a4788b36d43aaa0688d36c1160682" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 40845 } }
I20260806 03:47:03.295619 16573 raft_consensus.cc:388] T 00000000000000000000000000000000 P 466a4788b36d43aaa0688d36c1160682 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 03:47:03.295652 16573 raft_consensus.cc:482] T 00000000000000000000000000000000 P 466a4788b36d43aaa0688d36c1160682 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 03:47:03.295701 16573 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 466a4788b36d43aaa0688d36c1160682 [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:47:03.296378 16573 raft_consensus.cc:504] T 00000000000000000000000000000000 P 466a4788b36d43aaa0688d36c1160682 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "466a4788b36d43aaa0688d36c1160682" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 40845 } }
Aug 06 03:47:03 dist-test-slave-6ntk krb5kdc[16455](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988023, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.253.254@KRBTEST.COM
I20260806 03:47:03.296475 16573 leader_election.cc:302] T 00000000000000000000000000000000 P 466a4788b36d43aaa0688d36c1160682 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 466a4788b36d43aaa0688d36c1160682; no voters:
I20260806 03:47:03.296670 16573 leader_election.cc:288] T 00000000000000000000000000000000 P 466a4788b36d43aaa0688d36c1160682 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 03:47:03.296731 16578 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 466a4788b36d43aaa0688d36c1160682 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 03:47:03.296903 16578 raft_consensus.cc:686] T 00000000000000000000000000000000 P 466a4788b36d43aaa0688d36c1160682 [term 1 LEADER]: Becoming Leader. State: Replica: 466a4788b36d43aaa0688d36c1160682, State: Running, Role: LEADER
I20260806 03:47:03.296948 16573 sys_catalog.cc:564] T 00000000000000000000000000000000 P 466a4788b36d43aaa0688d36c1160682 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 03:47:03.297096 16578 consensus_queue.cc:237] T 00000000000000000000000000000000 P 466a4788b36d43aaa0688d36c1160682 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "466a4788b36d43aaa0688d36c1160682" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 40845 } }
I20260806 03:47:03.298142 16579 sys_catalog.cc:455] T 00000000000000000000000000000000 P 466a4788b36d43aaa0688d36c1160682 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "466a4788b36d43aaa0688d36c1160682" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "466a4788b36d43aaa0688d36c1160682" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 40845 } } }
I20260806 03:47:03.298267 16579 sys_catalog.cc:458] T 00000000000000000000000000000000 P 466a4788b36d43aaa0688d36c1160682 [sys.catalog]: This master's current role is: LEADER
I20260806 03:47:03.298425 16581 sys_catalog.cc:455] T 00000000000000000000000000000000 P 466a4788b36d43aaa0688d36c1160682 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 466a4788b36d43aaa0688d36c1160682. Latest consensus state: current_term: 1 leader_uuid: "466a4788b36d43aaa0688d36c1160682" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "466a4788b36d43aaa0688d36c1160682" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 40845 } } }
I20260806 03:47:03.298491 16581 sys_catalog.cc:458] T 00000000000000000000000000000000 P 466a4788b36d43aaa0688d36c1160682 [sys.catalog]: This master's current role is: LEADER
I20260806 03:47:03.298616 16589 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 03:47:03.299130 16589 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 03:47:03.300886 16589 catalog_manager.cc:1347] Generated new cluster ID: 1c9153b3acc44508b541b42df992cca5
I20260806 03:47:03.300944 16589 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 03:47:03.301720 16576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:03.289974 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32790 (local address 127.11.253.254:40845)
0806 03:47:03.290174 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0806 03:47:03.290181 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 03:47:03.290207 (+ 26us) server_negotiation.cc:407] Connection header received
0806 03:47:03.290602 (+ 395us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:47:03.290608 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:47:03.290685 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:47:03.290818 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:47:03.291683 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:03.292697 (+ 1014us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:03.293723 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:03.294193 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:03.294281 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:03.297079 (+ 2798us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:47:03.297083 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:47:03.297083 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:47:03.297095 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:47:03.299679 (+ 2584us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:47:03.300114 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:47:03.300117 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:47:03.300118 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:47:03.300241 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:47:03.300449 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:47:03.300451 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:47:03.300451 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:47:03.300772 (+ 321us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:47:03.301306 (+ 534us) server_negotiation.cc:1035] Waiting for connection context
0806 03:47:03.301384 (+ 78us) server_negotiation.cc:299] Negotiation successful
0806 03:47:03.301519 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":143,"thread_start_us":99,"threads_started":1}
I20260806 03:47:03.306535 16589 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 03:47:03.307264 16589 catalog_manager.cc:1504] Loading token signing keys...
I20260806 03:47:03.313575 16589 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 466a4788b36d43aaa0688d36c1160682: Generated new TSK 0
I20260806 03:47:03.313799 16589 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.253.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.253.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.253.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.193.keytab.
Entry for principal kudu/127.11.253.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.193.keytab.
I20260806 03:47:03.345281 12279 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskToVWRe/build/release/bin/kudu
/tmp/dist-test-taskToVWRe/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785988004637995-12279-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785988004637995-12279-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785988004637995-12279-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785988004637995-12279-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.253.193:0
--local_ip_for_outbound_sockets=127.11.253.193
--webserver_interface=127.11.253.193
--webserver_port=0
--tserver_master_addrs=127.11.253.254:40845
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.193.keytab
--principal=kudu/127.11.253.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.253.212:43285
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785988004637995-12279-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785988004637995-12279-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785988004637995-12279-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:47:03.429806 16605 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:47:03.430075 16605 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:47:03.430115 16605 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:47:03.431263 16605 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:47:03.431334 16605 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:47:03.431380 16605 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.253.193
I20260806 03:47:03.432847 16605 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.253.212:43285
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785988004637995-12279-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785988004637995-12279-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.193.keytab
--principal=kudu/127.11.253.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.253.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785988004637995-12279-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.253.193
--webserver_port=0
--tserver_master_addrs=127.11.253.254:40845
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16605
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.253.193
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785988004637995-12279-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:47:03.433106 16605 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:47:03.433346 16605 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:47:03.439469 16611 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:47:03.439492 16614 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:47:03.439630 16610 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:47:03.439888 16605 server_base.cc:1034] running on GCE node
I20260806 03:47:03.440081 16605 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:47:03.440349 16605 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:47:03.441519 16605 hybrid_clock.cc:648] HybridClock initialized: now 1785988023441506 us; error 33 us; skew 500 ppm
Aug 06 03:47:03 dist-test-slave-6ntk krb5kdc[16455](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988023, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.253.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:47:03.444375 16605 init.cc:377] Logged in from keytab as kudu/127.11.253.193@KRBTEST.COM (short username kudu)
I20260806 03:47:03.444940 16605 webserver.cc:460] Webserver started at http://127.11.253.193:40493/ using document root <none> and password file <none>
I20260806 03:47:03.445204 16605 fs_manager.cc:359] Metadata directory not provided
I20260806 03:47:03.445279 16605 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:47:03.445449 16605 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:47:03.446795 16605 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785988004637995-12279-0/minicluster-data/ts-0/data/instance:
uuid: "1aa56a17ccd64127a8fdfd3bde5c2f01"
format_stamp: "Formatted at 2026-08-06 03:47:03 on dist-test-slave-6ntk"
server_key: "63b34a551a4c7feb77b6a435289ded0b"
server_key_iv: "25c4d8d6619572ae94f73a6bdd7f4464"
server_key_version: "encryptionkey@0"
I20260806 03:47:03.447228 16605 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785988004637995-12279-0/minicluster-data/ts-0/wal/instance:
uuid: "1aa56a17ccd64127a8fdfd3bde5c2f01"
format_stamp: "Formatted at 2026-08-06 03:47:03 on dist-test-slave-6ntk"
server_key: "63b34a551a4c7feb77b6a435289ded0b"
server_key_iv: "25c4d8d6619572ae94f73a6bdd7f4464"
server_key_version: "encryptionkey@0"
I20260806 03:47:03.448798 16605 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 03:47:03.449620 16627 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:47:03.449813 16605 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 03:47:03.449960 16605 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785988004637995-12279-0/minicluster-data/ts-0/data,/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785988004637995-12279-0/minicluster-data/ts-0/wal
uuid: "1aa56a17ccd64127a8fdfd3bde5c2f01"
format_stamp: "Formatted at 2026-08-06 03:47:03 on dist-test-slave-6ntk"
server_key: "63b34a551a4c7feb77b6a435289ded0b"
server_key_iv: "25c4d8d6619572ae94f73a6bdd7f4464"
server_key_version: "encryptionkey@0"
I20260806 03:47:03.450048 16605 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785988004637995-12279-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785988004637995-12279-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785988004637995-12279-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:47:03.464283 16605 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:47:03.466413 16605 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:47:03.466610 16605 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:47:03.466890 16605 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 03:47:03.467248 16605 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 03:47:03.467312 16605 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 03:47:03.467358 16605 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 03:47:03.467391 16605 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 03:47:03.474087 16605 rpc_server.cc:307] RPC server started. Bound to: 127.11.253.193:41171
I20260806 03:47:03.474146 16799 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.253.193:41171 every 8 connection(s)
I20260806 03:47:03.474493 16605 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785988004637995-12279-0/minicluster-data/ts-0/data/info.pb
Aug 06 03:47:03 dist-test-slave-6ntk krb5kdc[16455](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988023, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.253.193@KRBTEST.COM for kudu/127.11.253.254@KRBTEST.COM
I20260806 03:47:03.481534 12279 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskToVWRe/build/release/bin/kudu as pid 16605
I20260806 03:47:03.481657 12279 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785988004637995-12279-0/minicluster-data/ts-0/wal/instance
I20260806 03:47:03.481948 12279 external_mini_cluster.cc:1428] Setting key 4999607f306655c15d9c8e1f02b7c721
I20260806 03:47:03.484467 16576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:03.475214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.253.193:46111 (local address 127.11.253.254:40845)
0806 03:47:03.475306 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0806 03:47:03.475308 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:47:03.475485 (+ 177us) server_negotiation.cc:407] Connection header received
0806 03:47:03.475996 (+ 511us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:47:03.475997 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:47:03.476007 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:47:03.476061 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:47:03.477059 (+ 998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:03.477574 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:03.478715 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:03.478923 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:03.478974 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:03.481243 (+ 2269us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:47:03.481245 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:47:03.481245 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:47:03.481252 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:47:03.483042 (+ 1790us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:47:03.483439 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:47:03.483440 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:47:03.483441 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:47:03.483509 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:47:03.483740 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:47:03.483741 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:47:03.483742 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:47:03.483882 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:47:03.483950 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0806 03:47:03.484216 (+ 266us) server_negotiation.cc:299] Negotiation successful
0806 03:47:03.484318 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260806 03:47:03.484617 16802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:03.475170 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:40845 (local address 127.11.253.193:46111)
0806 03:47:03.475347 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0806 03:47:03.475388 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 03:47:03.475906 (+ 518us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:47:03.476119 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:47:03.476123 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:47:03.476759 (+ 636us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:47:03.476970 (+ 211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:03.476975 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:03.477666 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:03.477667 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:03.478636 (+ 969us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:03.478639 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:03.479001 (+ 362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:03.479002 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:03.479134 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:03.479683 (+ 549us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:47:03.479692 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:47:03.481131 (+ 1439us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:47:03.483129 (+ 1998us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:47:03.483136 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:47:03.483138 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:47:03.483363 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:47:03.483576 (+ 213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:47:03.483577 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:47:03.483577 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:47:03.483679 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:47:03.483946 (+ 267us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:47:03.483951 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:47:03.484065 (+ 114us) client_negotiation.cc:769] Sending connection context
0806 03:47:03.484182 (+ 117us) client_negotiation.cc:240] Negotiation successful
0806 03:47:03.484330 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":100,"threads_started":1}
I20260806 03:47:03.485304 16800 heartbeater.cc:347] Connected to a master server at 127.11.253.254:40845
I20260806 03:47:03.485440 16800 heartbeater.cc:464] Registering TS with master...
I20260806 03:47:03.485627 16800 heartbeater.cc:511] Master 127.11.253.254:40845 requested a full tablet report, sending...
I20260806 03:47:03.486178 16521 ts_manager.cc:194] Registered new tserver with Master: 1aa56a17ccd64127a8fdfd3bde5c2f01 (127.11.253.193:41171)
I20260806 03:47:03.486796 16521 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.253.193@KRBTEST.COM'} at 127.11.253.193:46111
WARNING: no policy specified for kudu/127.11.253.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.253.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.253.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.194.keytab.
Entry for principal kudu/127.11.253.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.194.keytab.
I20260806 03:47:03.512831 12279 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskToVWRe/build/release/bin/kudu
/tmp/dist-test-taskToVWRe/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785988004637995-12279-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785988004637995-12279-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785988004637995-12279-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785988004637995-12279-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.253.194:0
--local_ip_for_outbound_sockets=127.11.253.194
--webserver_interface=127.11.253.194
--webserver_port=0
--tserver_master_addrs=127.11.253.254:40845
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.194.keytab
--principal=kudu/127.11.253.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.253.212:43285
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785988004637995-12279-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785988004637995-12279-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785988004637995-12279-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:47:03.600924 16810 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:47:03.601176 16810 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:47:03.601227 16810 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:47:03.602473 16810 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:47:03.602545 16810 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:47:03.602586 16810 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.253.194
I20260806 03:47:03.604126 16810 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.253.212:43285
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785988004637995-12279-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785988004637995-12279-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.194.keytab
--principal=kudu/127.11.253.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.253.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785988004637995-12279-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.253.194
--webserver_port=0
--tserver_master_addrs=127.11.253.254:40845
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16810
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.253.194
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785988004637995-12279-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:47:03.604419 16810 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:47:03.604664 16810 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:47:03.609912 16821 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:47:03.609912 16823 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:47:03.609938 16825 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:47:03.610213 16810 server_base.cc:1034] running on GCE node
I20260806 03:47:03.610426 16810 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:47:03.610682 16810 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:47:03.611848 16810 hybrid_clock.cc:648] HybridClock initialized: now 1785988023611885 us; error 81 us; skew 500 ppm
Aug 06 03:47:03 dist-test-slave-6ntk krb5kdc[16455](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988023, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.253.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:47:03.614566 16810 init.cc:377] Logged in from keytab as kudu/127.11.253.194@KRBTEST.COM (short username kudu)
I20260806 03:47:03.615027 16810 webserver.cc:460] Webserver started at http://127.11.253.194:40675/ using document root <none> and password file <none>
I20260806 03:47:03.615278 16810 fs_manager.cc:359] Metadata directory not provided
I20260806 03:47:03.615355 16810 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:47:03.615512 16810 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:47:03.616676 16810 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785988004637995-12279-0/minicluster-data/ts-1/data/instance:
uuid: "32187df18f9a4aaea97d7b559448f995"
format_stamp: "Formatted at 2026-08-06 03:47:03 on dist-test-slave-6ntk"
server_key: "af98727d617a4059feec56264c2acb5d"
server_key_iv: "a5eda2ec645be1c62bcbb2c8aa09fe25"
server_key_version: "encryptionkey@0"
I20260806 03:47:03.617059 16810 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785988004637995-12279-0/minicluster-data/ts-1/wal/instance:
uuid: "32187df18f9a4aaea97d7b559448f995"
format_stamp: "Formatted at 2026-08-06 03:47:03 on dist-test-slave-6ntk"
server_key: "af98727d617a4059feec56264c2acb5d"
server_key_iv: "a5eda2ec645be1c62bcbb2c8aa09fe25"
server_key_version: "encryptionkey@0"
I20260806 03:47:03.618595 16810 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 03:47:03.619514 16834 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:47:03.619736 16810 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 03:47:03.619859 16810 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785988004637995-12279-0/minicluster-data/ts-1/data,/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785988004637995-12279-0/minicluster-data/ts-1/wal
uuid: "32187df18f9a4aaea97d7b559448f995"
format_stamp: "Formatted at 2026-08-06 03:47:03 on dist-test-slave-6ntk"
server_key: "af98727d617a4059feec56264c2acb5d"
server_key_iv: "a5eda2ec645be1c62bcbb2c8aa09fe25"
server_key_version: "encryptionkey@0"
I20260806 03:47:03.619942 16810 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785988004637995-12279-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785988004637995-12279-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785988004637995-12279-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:47:03.644018 16810 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:47:03.646405 16810 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:47:03.646581 16810 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:47:03.646839 16810 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 03:47:03.647168 16810 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 03:47:03.647223 16810 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 03:47:03.647270 16810 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 03:47:03.647292 16810 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 03:47:03.653350 16810 rpc_server.cc:307] RPC server started. Bound to: 127.11.253.194:43661
I20260806 03:47:03.653432 16997 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.253.194:43661 every 8 connection(s)
I20260806 03:47:03.653764 16810 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785988004637995-12279-0/minicluster-data/ts-1/data/info.pb
I20260806 03:47:03.659191 12279 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskToVWRe/build/release/bin/kudu as pid 16810
I20260806 03:47:03.659369 12279 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785988004637995-12279-0/minicluster-data/ts-1/wal/instance
I20260806 03:47:03.659595 12279 external_mini_cluster.cc:1428] Setting key 85b258574b506a73d4c67c0c6600e177
Aug 06 03:47:03 dist-test-slave-6ntk krb5kdc[16455](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988023, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.253.194@KRBTEST.COM for kudu/127.11.253.254@KRBTEST.COM
I20260806 03:47:03.663709 16576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:03.654604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.253.194:46869 (local address 127.11.253.254:40845)
0806 03:47:03.654664 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0806 03:47:03.654666 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:47:03.654929 (+ 263us) server_negotiation.cc:407] Connection header received
0806 03:47:03.655456 (+ 527us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:47:03.655457 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:47:03.655467 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:47:03.655527 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:47:03.656443 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:03.656937 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:03.657986 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:03.658180 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:03.658239 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:03.660766 (+ 2527us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:47:03.660767 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:47:03.660768 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:47:03.660776 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:47:03.662232 (+ 1456us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:47:03.662710 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:47:03.662711 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:47:03.662712 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:47:03.662777 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:47:03.663045 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:47:03.663046 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:47:03.663046 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:47:03.663163 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:47:03.663235 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0806 03:47:03.663522 (+ 287us) server_negotiation.cc:299] Negotiation successful
0806 03:47:03.663620 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260806 03:47:03.663918 17002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:03.654585 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:40845 (local address 127.11.253.194:46869)
0806 03:47:03.654792 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0806 03:47:03.654836 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0806 03:47:03.655369 (+ 533us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:47:03.655582 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:47:03.655585 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:47:03.656170 (+ 585us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:47:03.656368 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:03.656373 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:03.657026 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:03.657026 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:03.657834 (+ 808us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:03.657836 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:03.658258 (+ 422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:03.658259 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:03.658363 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:03.658941 (+ 578us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:47:03.658949 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:47:03.660640 (+ 1691us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:47:03.662382 (+ 1742us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:47:03.662390 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:47:03.662392 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:47:03.662627 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:47:03.662851 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:47:03.662851 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:47:03.662852 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:47:03.662969 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:47:03.663237 (+ 268us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:47:03.663245 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:47:03.663386 (+ 141us) client_negotiation.cc:769] Sending connection context
0806 03:47:03.663500 (+ 114us) client_negotiation.cc:240] Negotiation successful
0806 03:47:03.663671 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":101,"threads_started":1}
I20260806 03:47:03.664422 16998 heartbeater.cc:347] Connected to a master server at 127.11.253.254:40845
I20260806 03:47:03.664563 16998 heartbeater.cc:464] Registering TS with master...
I20260806 03:47:03.664742 16998 heartbeater.cc:511] Master 127.11.253.254:40845 requested a full tablet report, sending...
I20260806 03:47:03.665189 16521 ts_manager.cc:194] Registered new tserver with Master: 32187df18f9a4aaea97d7b559448f995 (127.11.253.194:43661)
I20260806 03:47:03.665565 16521 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.253.194@KRBTEST.COM'} at 127.11.253.194:46869
WARNING: no policy specified for kudu/127.11.253.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.253.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.253.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.195.keytab.
Entry for principal kudu/127.11.253.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.195.keytab.
I20260806 03:47:03.688079 12279 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskToVWRe/build/release/bin/kudu
/tmp/dist-test-taskToVWRe/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785988004637995-12279-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785988004637995-12279-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785988004637995-12279-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785988004637995-12279-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.253.195:0
--local_ip_for_outbound_sockets=127.11.253.195
--webserver_interface=127.11.253.195
--webserver_port=0
--tserver_master_addrs=127.11.253.254:40845
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.195.keytab
--principal=kudu/127.11.253.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.253.212:43285
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785988004637995-12279-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785988004637995-12279-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785988004637995-12279-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:47:03.772828 17005 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:47:03.773057 17005 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:47:03.773083 17005 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:47:03.774354 17005 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:47:03.774398 17005 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:47:03.774426 17005 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.253.195
I20260806 03:47:03.775931 17005 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.253.212:43285
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785988004637995-12279-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785988004637995-12279-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.195.keytab
--principal=kudu/127.11.253.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.253.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785988004637995-12279-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.253.195
--webserver_port=0
--tserver_master_addrs=127.11.253.254:40845
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17005
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.253.195
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785988004637995-12279-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:47:03.776183 17005 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:47:03.776443 17005 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:47:03.781813 17015 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:47:03.781843 17014 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:47:03.781946 17005 server_base.cc:1034] running on GCE node
W20260806 03:47:03.781813 17017 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:47:03.782349 17005 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:47:03.782621 17005 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:47:03.783797 17005 hybrid_clock.cc:648] HybridClock initialized: now 1785988023783773 us; error 38 us; skew 500 ppm
Aug 06 03:47:03 dist-test-slave-6ntk krb5kdc[16455](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988023, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.253.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:47:03.786619 17005 init.cc:377] Logged in from keytab as kudu/127.11.253.195@KRBTEST.COM (short username kudu)
I20260806 03:47:03.787127 17005 webserver.cc:460] Webserver started at http://127.11.253.195:36089/ using document root <none> and password file <none>
I20260806 03:47:03.787371 17005 fs_manager.cc:359] Metadata directory not provided
I20260806 03:47:03.787446 17005 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:47:03.787619 17005 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:47:03.788712 17005 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785988004637995-12279-0/minicluster-data/ts-2/data/instance:
uuid: "8a6005031e9043d18d7a62ded7b6d218"
format_stamp: "Formatted at 2026-08-06 03:47:03 on dist-test-slave-6ntk"
server_key: "02f39c8f3ff520f8a71192195128321d"
server_key_iv: "fdc519c26a35b8642c1d135019233e1c"
server_key_version: "encryptionkey@0"
I20260806 03:47:03.789139 17005 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785988004637995-12279-0/minicluster-data/ts-2/wal/instance:
uuid: "8a6005031e9043d18d7a62ded7b6d218"
format_stamp: "Formatted at 2026-08-06 03:47:03 on dist-test-slave-6ntk"
server_key: "02f39c8f3ff520f8a71192195128321d"
server_key_iv: "fdc519c26a35b8642c1d135019233e1c"
server_key_version: "encryptionkey@0"
I20260806 03:47:03.790804 17005 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 03:47:03.791646 17031 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:47:03.791831 17005 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 03:47:03.791953 17005 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785988004637995-12279-0/minicluster-data/ts-2/data,/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785988004637995-12279-0/minicluster-data/ts-2/wal
uuid: "8a6005031e9043d18d7a62ded7b6d218"
format_stamp: "Formatted at 2026-08-06 03:47:03 on dist-test-slave-6ntk"
server_key: "02f39c8f3ff520f8a71192195128321d"
server_key_iv: "fdc519c26a35b8642c1d135019233e1c"
server_key_version: "encryptionkey@0"
I20260806 03:47:03.792030 17005 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785988004637995-12279-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785988004637995-12279-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785988004637995-12279-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:47:03.807482 17005 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:47:03.809693 17005 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:47:03.809927 17005 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:47:03.810202 17005 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 03:47:03.810544 17005 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 03:47:03.810600 17005 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 03:47:03.810647 17005 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 03:47:03.810672 17005 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 03:47:03.816872 17005 rpc_server.cc:307] RPC server started. Bound to: 127.11.253.195:40803
I20260806 03:47:03.816911 17202 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.253.195:40803 every 8 connection(s)
I20260806 03:47:03.817243 17005 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785988004637995-12279-0/minicluster-data/ts-2/data/info.pb
I20260806 03:47:03.823511 12279 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskToVWRe/build/release/bin/kudu as pid 17005
Aug 06 03:47:03 dist-test-slave-6ntk krb5kdc[16455](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988023, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.253.195@KRBTEST.COM for kudu/127.11.253.254@KRBTEST.COM
I20260806 03:47:03.823627 12279 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785988004637995-12279-0/minicluster-data/ts-2/wal/instance
I20260806 03:47:03.823843 12279 external_mini_cluster.cc:1428] Setting key 28d9b6a515df0ad28d3bb8337b021837
I20260806 03:47:03.827272 16576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:03.817996 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.253.195:43675 (local address 127.11.253.254:40845)
0806 03:47:03.818065 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0806 03:47:03.818067 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:47:03.818344 (+ 277us) server_negotiation.cc:407] Connection header received
0806 03:47:03.818882 (+ 538us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:47:03.818884 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:47:03.818897 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:47:03.818956 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:47:03.819975 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:03.820487 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:03.821516 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:03.821733 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:03.821812 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:03.824236 (+ 2424us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:47:03.824238 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:47:03.824238 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:47:03.824247 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:47:03.825763 (+ 1516us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:47:03.826277 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:47:03.826278 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:47:03.826279 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:47:03.826349 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:47:03.826609 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:47:03.826610 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:47:03.826611 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:47:03.826740 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:47:03.826800 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0806 03:47:03.827071 (+ 271us) server_negotiation.cc:299] Negotiation successful
0806 03:47:03.827148 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260806 03:47:03.827309 17207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:03.818006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:40845 (local address 127.11.253.195:43675)
0806 03:47:03.818183 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0806 03:47:03.818222 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0806 03:47:03.818755 (+ 533us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:47:03.819030 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:47:03.819035 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:47:03.819644 (+ 609us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:47:03.819856 (+ 212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:03.819861 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:03.820585 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:03.820586 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:03.821409 (+ 823us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:03.821411 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:03.821820 (+ 409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:03.821822 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:03.821982 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:03.822554 (+ 572us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:47:03.822563 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:47:03.824120 (+ 1557us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:47:03.825849 (+ 1729us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:47:03.825855 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:47:03.825857 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:47:03.826163 (+ 306us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:47:03.826429 (+ 266us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:47:03.826429 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:47:03.826430 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:47:03.826534 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:47:03.826800 (+ 266us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:47:03.826804 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:47:03.826936 (+ 132us) client_negotiation.cc:769] Sending connection context
0806 03:47:03.827015 (+ 79us) client_negotiation.cc:240] Negotiation successful
0806 03:47:03.827140 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":85,"threads_started":1}
I20260806 03:47:03.828145 17205 heartbeater.cc:347] Connected to a master server at 127.11.253.254:40845
I20260806 03:47:03.828287 17205 heartbeater.cc:464] Registering TS with master...
I20260806 03:47:03.828457 17205 heartbeater.cc:511] Master 127.11.253.254:40845 requested a full tablet report, sending...
I20260806 03:47:03.828841 16521 ts_manager.cc:194] Registered new tserver with Master: 8a6005031e9043d18d7a62ded7b6d218 (127.11.253.195:40803)
I20260806 03:47:03.829226 16521 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.253.195@KRBTEST.COM'} at 127.11.253.195:43675
I20260806 03:47:03.835708 12279 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 06 03:47:03 dist-test-slave-6ntk krb5kdc[16455](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988023, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 06 03:47:03 dist-test-slave-6ntk krb5kdc[16455](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988023, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.11.253.254@KRBTEST.COM
I20260806 03:47:03.860862 16576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:03.853579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32804 (local address 127.11.253.254:40845)
0806 03:47:03.853647 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0806 03:47:03.853649 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:47:03.853726 (+ 77us) server_negotiation.cc:407] Connection header received
0806 03:47:03.853773 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:47:03.853774 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:47:03.853783 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:47:03.853843 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:47:03.854524 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:03.855026 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:03.855753 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:03.855937 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:03.855990 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:03.858230 (+ 2240us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:47:03.858231 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:47:03.858232 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:47:03.858238 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:47:03.859759 (+ 1521us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:47:03.860056 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:47:03.860056 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:47:03.860057 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:47:03.860109 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:47:03.860276 (+ 167us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:47:03.860277 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:47:03.860277 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:47:03.860431 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:47:03.860508 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0806 03:47:03.860593 (+ 85us) server_negotiation.cc:299] Negotiation successful
0806 03:47:03.860651 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
W20260806 03:47:03.861238 16521 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:32804
I20260806 03:47:03.861712 12279 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskToVWRe/build/release/bin/kudu with pid 16605
I20260806 03:47:03.874679 12279 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskToVWRe/build/release/bin/kudu with pid 16810
I20260806 03:47:03.886761 12279 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskToVWRe/build/release/bin/kudu with pid 17005
I20260806 03:47:03.895921 12279 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskToVWRe/build/release/bin/kudu with pid 16472
2026-08-06T03:47:03Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (2114 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785988004637995-12279-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 03:47:03 dist-test-slave-6ntk krb5kdc[17226](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 03:47:03 dist-test-slave-6ntk krb5kdc[17226](info): set up 2 sockets
Aug 06 03:47:03 dist-test-slave-6ntk krb5kdc[17226](info): commencing operation
krb5kdc: starting...
W20260806 03:47:05.120661 12279 mini_kdc.cc:121] Time spent starting KDC: real 1.200s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785988004637995-12279-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785988004637995-12279-0/krb5kdc/test-user.keytab.
Aug 06 03:47:05 dist-test-slave-6ntk krb5kdc[17226](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988025, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T03:47:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T03:47:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.253.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.253.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.253.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.254.keytab.
Entry for principal kudu/127.11.253.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.254.keytab.
I20260806 03:47:05.268983 12279 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskToVWRe/build/release/bin/kudu
/tmp/dist-test-taskToVWRe/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785988004637995-12279-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785988004637995-12279-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785988004637995-12279-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785988004637995-12279-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.253.254:40031
--webserver_interface=127.11.253.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.254.keytab
--principal=kudu/127.11.253.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.253.212:34987
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.253.254:40031
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785988004637995-12279-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785988004637995-12279-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785988004637995-12279-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:47:05.356513 17242 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:47:05.356755 17242 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:47:05.356781 17242 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:47:05.358037 17242 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 03:47:05.358089 17242 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:47:05.358103 17242 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:47:05.358116 17242 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 03:47:05.358129 17242 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 03:47:05.359730 17242 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.253.212:34987
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785988004637995-12279-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785988004637995-12279-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.253.254:40031
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.254.keytab
--principal=kudu/127.11.253.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.253.254:40031
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785988004637995-12279-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.253.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17242
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785988004637995-12279-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:47:05.359958 17242 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:47:05.360149 17242 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:47:05.365278 17251 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:47:05.365283 17254 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:47:05.365309 17252 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:47:05.365478 17242 server_base.cc:1034] running on GCE node
I20260806 03:47:05.365859 17242 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:47:05.366201 17242 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:47:05.367367 17242 hybrid_clock.cc:648] HybridClock initialized: now 1785988025367339 us; error 46 us; skew 500 ppm
Aug 06 03:47:05 dist-test-slave-6ntk krb5kdc[17226](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988025, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.253.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:47:05.369997 17242 init.cc:377] Logged in from keytab as kudu/127.11.253.254@KRBTEST.COM (short username kudu)
I20260806 03:47:05.370404 17242 webserver.cc:460] Webserver started at http://127.11.253.254:41615/ using document root <none> and password file <none>
I20260806 03:47:05.370610 17242 fs_manager.cc:359] Metadata directory not provided
I20260806 03:47:05.370651 17242 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:47:05.370743 17242 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:47:05.371788 17242 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785988004637995-12279-0/minicluster-data/master-0/data/instance:
uuid: "8365632140694b1bbdebc81a73d08e2a"
format_stamp: "Formatted at 2026-08-06 03:47:05 on dist-test-slave-6ntk"
server_key: "760cc9853ff1b1b3bf8bbc26af168b2b"
server_key_iv: "c74c54bff8158326d9036084f13e5360"
server_key_version: "encryptionkey@0"
I20260806 03:47:05.372089 17242 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785988004637995-12279-0/minicluster-data/master-0/wal/instance:
uuid: "8365632140694b1bbdebc81a73d08e2a"
format_stamp: "Formatted at 2026-08-06 03:47:05 on dist-test-slave-6ntk"
server_key: "760cc9853ff1b1b3bf8bbc26af168b2b"
server_key_iv: "c74c54bff8158326d9036084f13e5360"
server_key_version: "encryptionkey@0"
I20260806 03:47:05.373407 17242 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 03:47:05.374244 17267 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:47:05.374459 17242 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 03:47:05.374575 17242 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785988004637995-12279-0/minicluster-data/master-0/data,/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785988004637995-12279-0/minicluster-data/master-0/wal
uuid: "8365632140694b1bbdebc81a73d08e2a"
format_stamp: "Formatted at 2026-08-06 03:47:05 on dist-test-slave-6ntk"
server_key: "760cc9853ff1b1b3bf8bbc26af168b2b"
server_key_iv: "c74c54bff8158326d9036084f13e5360"
server_key_version: "encryptionkey@0"
I20260806 03:47:05.374656 17242 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785988004637995-12279-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785988004637995-12279-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785988004637995-12279-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:47:05.394381 17242 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:47:05.396557 17242 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:47:05.396736 17242 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:47:05.400894 17242 rpc_server.cc:307] RPC server started. Bound to: 127.11.253.254:40031
I20260806 03:47:05.400934 17344 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.253.254:40031 every 8 connection(s)
I20260806 03:47:05.401465 17242 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785988004637995-12279-0/minicluster-data/master-0/data/info.pb
I20260806 03:47:05.402232 17348 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:47:05.404628 17348 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8365632140694b1bbdebc81a73d08e2a: Bootstrap starting.
I20260806 03:47:05.405297 12279 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskToVWRe/build/release/bin/kudu as pid 17242
I20260806 03:47:05.405331 17348 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8365632140694b1bbdebc81a73d08e2a: Neither blocks nor log segments found. Creating new log.
I20260806 03:47:05.405428 12279 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785988004637995-12279-0/minicluster-data/master-0/wal/instance
I20260806 03:47:05.405649 12279 external_mini_cluster.cc:1428] Setting key 5c26e3af15db9b9995a1960c853ca101
I20260806 03:47:05.405704 17348 log.cc:824] T 00000000000000000000000000000000 P 8365632140694b1bbdebc81a73d08e2a: Log is configured to *not* fsync() on all Append() calls
I20260806 03:47:05.406713 17348 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8365632140694b1bbdebc81a73d08e2a: No bootstrap required, opened a new log
I20260806 03:47:05.409302 17348 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8365632140694b1bbdebc81a73d08e2a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8365632140694b1bbdebc81a73d08e2a" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 40031 } }
I20260806 03:47:05.409495 17348 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8365632140694b1bbdebc81a73d08e2a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:47:05.409533 17348 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8365632140694b1bbdebc81a73d08e2a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8365632140694b1bbdebc81a73d08e2a, State: Initialized, Role: FOLLOWER
I20260806 03:47:05.409663 17348 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8365632140694b1bbdebc81a73d08e2a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8365632140694b1bbdebc81a73d08e2a" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 40031 } }
I20260806 03:47:05.409740 17348 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8365632140694b1bbdebc81a73d08e2a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 03:47:05.409770 17348 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8365632140694b1bbdebc81a73d08e2a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 03:47:05.409811 17348 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8365632140694b1bbdebc81a73d08e2a [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:47:05.410548 17348 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8365632140694b1bbdebc81a73d08e2a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8365632140694b1bbdebc81a73d08e2a" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 40031 } }
I20260806 03:47:05.410682 17348 leader_election.cc:302] T 00000000000000000000000000000000 P 8365632140694b1bbdebc81a73d08e2a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8365632140694b1bbdebc81a73d08e2a; no voters:
I20260806 03:47:05.410890 17348 leader_election.cc:288] T 00000000000000000000000000000000 P 8365632140694b1bbdebc81a73d08e2a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 03:47:05.410943 17356 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8365632140694b1bbdebc81a73d08e2a [term 1 FOLLOWER]: Leader election won for term 1
I20260806 03:47:05.411196 17356 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8365632140694b1bbdebc81a73d08e2a [term 1 LEADER]: Becoming Leader. State: Replica: 8365632140694b1bbdebc81a73d08e2a, State: Running, Role: LEADER
I20260806 03:47:05.411293 17348 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8365632140694b1bbdebc81a73d08e2a [sys.catalog]: configured and running, proceeding with master startup.
I20260806 03:47:05.411350 17356 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8365632140694b1bbdebc81a73d08e2a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8365632140694b1bbdebc81a73d08e2a" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 40031 } }
I20260806 03:47:05.411744 17356 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8365632140694b1bbdebc81a73d08e2a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8365632140694b1bbdebc81a73d08e2a. Latest consensus state: current_term: 1 leader_uuid: "8365632140694b1bbdebc81a73d08e2a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8365632140694b1bbdebc81a73d08e2a" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 40031 } } }
I20260806 03:47:05.411752 17357 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8365632140694b1bbdebc81a73d08e2a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8365632140694b1bbdebc81a73d08e2a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8365632140694b1bbdebc81a73d08e2a" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 40031 } } }
I20260806 03:47:05.411887 17356 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8365632140694b1bbdebc81a73d08e2a [sys.catalog]: This master's current role is: LEADER
I20260806 03:47:05.411917 17357 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8365632140694b1bbdebc81a73d08e2a [sys.catalog]: This master's current role is: LEADER
I20260806 03:47:05.412107 17367 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 03:47:05.412874 17367 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 06 03:47:05 dist-test-slave-6ntk krb5kdc[17226](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988025, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.253.254@KRBTEST.COM
I20260806 03:47:05.416167 17367 catalog_manager.cc:1347] Generated new cluster ID: 51a9a451d5b44e6ea62aebc89f9f549a
I20260806 03:47:05.416222 17367 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 03:47:05.417809 17353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:05.406268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40392 (local address 127.11.253.254:40031)
0806 03:47:05.406455 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0806 03:47:05.406467 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 03:47:05.406506 (+ 39us) server_negotiation.cc:407] Connection header received
0806 03:47:05.407091 (+ 585us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:47:05.407099 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:47:05.407159 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:47:05.407303 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:47:05.408188 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:05.409198 (+ 1010us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:05.410298 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:05.410782 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:05.410945 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:05.413991 (+ 3046us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:47:05.413995 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:47:05.413996 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:47:05.414009 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:47:05.415981 (+ 1972us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:47:05.416436 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:47:05.416440 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:47:05.416440 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:47:05.416557 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:47:05.416788 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:47:05.416789 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:47:05.416789 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:47:05.417122 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:47:05.417288 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0806 03:47:05.417410 (+ 122us) server_negotiation.cc:299] Negotiation successful
0806 03:47:05.417562 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":103,"threads_started":1}
I20260806 03:47:05.442246 17367 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 03:47:05.443025 17367 catalog_manager.cc:1504] Loading token signing keys...
I20260806 03:47:05.447319 17367 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8365632140694b1bbdebc81a73d08e2a: Generated new TSK 0
I20260806 03:47:05.447525 17367 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.253.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.253.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.253.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.193.keytab.
Entry for principal kudu/127.11.253.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.193.keytab.
I20260806 03:47:05.482081 12279 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskToVWRe/build/release/bin/kudu
/tmp/dist-test-taskToVWRe/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785988004637995-12279-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785988004637995-12279-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785988004637995-12279-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785988004637995-12279-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.253.193:0
--local_ip_for_outbound_sockets=127.11.253.193
--webserver_interface=127.11.253.193
--webserver_port=0
--tserver_master_addrs=127.11.253.254:40031
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.193.keytab
--principal=kudu/127.11.253.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.253.212:34987
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785988004637995-12279-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785988004637995-12279-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785988004637995-12279-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:47:05.572813 17389 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:47:05.573086 17389 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:47:05.573128 17389 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:47:05.574386 17389 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:47:05.574467 17389 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:47:05.574515 17389 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.253.193
I20260806 03:47:05.576170 17389 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.253.212:34987
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785988004637995-12279-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785988004637995-12279-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.193.keytab
--principal=kudu/127.11.253.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.253.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785988004637995-12279-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.253.193
--webserver_port=0
--tserver_master_addrs=127.11.253.254:40031
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17389
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.253.193
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785988004637995-12279-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:47:05.576471 17389 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:47:05.576723 17389 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:47:05.582301 17398 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:47:05.582350 17389 server_base.cc:1034] running on GCE node
W20260806 03:47:05.582314 17400 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:47:05.582322 17396 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:47:05.582906 17389 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:47:05.583176 17389 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:47:05.584376 17389 hybrid_clock.cc:648] HybridClock initialized: now 1785988025584367 us; error 42 us; skew 500 ppm
Aug 06 03:47:05 dist-test-slave-6ntk krb5kdc[17226](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988025, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.253.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:47:05.587199 17389 init.cc:377] Logged in from keytab as kudu/127.11.253.193@KRBTEST.COM (short username kudu)
I20260806 03:47:05.587766 17389 webserver.cc:460] Webserver started at http://127.11.253.193:43089/ using document root <none> and password file <none>
I20260806 03:47:05.588054 17389 fs_manager.cc:359] Metadata directory not provided
I20260806 03:47:05.588114 17389 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:47:05.588285 17389 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:47:05.589395 17389 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785988004637995-12279-0/minicluster-data/ts-0/data/instance:
uuid: "9276b3d22bef49dabad862d2135ecefd"
format_stamp: "Formatted at 2026-08-06 03:47:05 on dist-test-slave-6ntk"
server_key: "1ceb381f7eb80f736ac23872d8cad171"
server_key_iv: "285eabe081c16e346918bb830d84485c"
server_key_version: "encryptionkey@0"
I20260806 03:47:05.589772 17389 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785988004637995-12279-0/minicluster-data/ts-0/wal/instance:
uuid: "9276b3d22bef49dabad862d2135ecefd"
format_stamp: "Formatted at 2026-08-06 03:47:05 on dist-test-slave-6ntk"
server_key: "1ceb381f7eb80f736ac23872d8cad171"
server_key_iv: "285eabe081c16e346918bb830d84485c"
server_key_version: "encryptionkey@0"
I20260806 03:47:05.591264 17389 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 03:47:05.592068 17410 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:47:05.592253 17389 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 03:47:05.592397 17389 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785988004637995-12279-0/minicluster-data/ts-0/data,/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785988004637995-12279-0/minicluster-data/ts-0/wal
uuid: "9276b3d22bef49dabad862d2135ecefd"
format_stamp: "Formatted at 2026-08-06 03:47:05 on dist-test-slave-6ntk"
server_key: "1ceb381f7eb80f736ac23872d8cad171"
server_key_iv: "285eabe081c16e346918bb830d84485c"
server_key_version: "encryptionkey@0"
I20260806 03:47:05.592463 17389 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785988004637995-12279-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785988004637995-12279-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785988004637995-12279-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:47:05.614343 17389 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:47:05.617033 17389 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:47:05.617163 17389 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:47:05.617486 17389 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 03:47:05.617844 17389 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 03:47:05.617901 17389 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 03:47:05.617925 17389 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 03:47:05.617938 17389 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 03:47:05.624437 17389 rpc_server.cc:307] RPC server started. Bound to: 127.11.253.193:46007
I20260806 03:47:05.624495 17584 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.253.193:46007 every 8 connection(s)
I20260806 03:47:05.624830 17389 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785988004637995-12279-0/minicluster-data/ts-0/data/info.pb
I20260806 03:47:05.628782 12279 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskToVWRe/build/release/bin/kudu as pid 17389
I20260806 03:47:05.628878 12279 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785988004637995-12279-0/minicluster-data/ts-0/wal/instance
I20260806 03:47:05.629099 12279 external_mini_cluster.cc:1428] Setting key 36c112355492255940e81258f2e0fb5b
Aug 06 03:47:05 dist-test-slave-6ntk krb5kdc[17226](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988025, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.253.193@KRBTEST.COM for kudu/127.11.253.254@KRBTEST.COM
I20260806 03:47:05.635311 17353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:05.625591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.253.193:38089 (local address 127.11.253.254:40031)
0806 03:47:05.625646 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0806 03:47:05.625648 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:47:05.625977 (+ 329us) server_negotiation.cc:407] Connection header received
0806 03:47:05.626900 (+ 923us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:47:05.626901 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:47:05.626914 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:47:05.626984 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:47:05.627645 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:05.628187 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:05.629426 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:05.629689 (+ 263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:05.629765 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:05.632108 (+ 2343us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:47:05.632110 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:47:05.632110 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:47:05.632118 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:47:05.633627 (+ 1509us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:47:05.634212 (+ 585us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:47:05.634213 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:47:05.634213 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:47:05.634283 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:47:05.634577 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:47:05.634578 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:47:05.634578 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:47:05.634707 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:47:05.634797 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0806 03:47:05.635118 (+ 321us) server_negotiation.cc:299] Negotiation successful
0806 03:47:05.635215 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260806 03:47:05.635525 17590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:05.625556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:40031 (local address 127.11.253.193:38089)
0806 03:47:05.625737 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0806 03:47:05.625782 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 03:47:05.626780 (+ 998us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:47:05.627068 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:47:05.627071 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:47:05.627338 (+ 267us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:47:05.627538 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:05.627542 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:05.628333 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:05.628334 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:05.629320 (+ 986us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:05.629323 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:05.629804 (+ 481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:05.629805 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:05.629996 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:05.630581 (+ 585us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:47:05.630589 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:47:05.632002 (+ 1413us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:47:05.633714 (+ 1712us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:47:05.633722 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:47:05.633725 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:47:05.634133 (+ 408us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:47:05.634368 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:47:05.634369 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:47:05.634370 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:47:05.634499 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:47:05.634802 (+ 303us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:47:05.634810 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:47:05.634959 (+ 149us) client_negotiation.cc:769] Sending connection context
0806 03:47:05.635082 (+ 123us) client_negotiation.cc:240] Negotiation successful
0806 03:47:05.635266 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":101,"threads_started":1}
I20260806 03:47:05.636097 17587 heartbeater.cc:347] Connected to a master server at 127.11.253.254:40031
I20260806 03:47:05.636199 17587 heartbeater.cc:464] Registering TS with master...
I20260806 03:47:05.636420 17587 heartbeater.cc:511] Master 127.11.253.254:40031 requested a full tablet report, sending...
I20260806 03:47:05.637042 17293 ts_manager.cc:194] Registered new tserver with Master: 9276b3d22bef49dabad862d2135ecefd (127.11.253.193:46007)
I20260806 03:47:05.638082 17293 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.253.193@KRBTEST.COM'} at 127.11.253.193:38089
WARNING: no policy specified for kudu/127.11.253.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.253.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.253.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.194.keytab.
Entry for principal kudu/127.11.253.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.194.keytab.
I20260806 03:47:05.661021 12279 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskToVWRe/build/release/bin/kudu
/tmp/dist-test-taskToVWRe/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785988004637995-12279-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785988004637995-12279-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785988004637995-12279-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785988004637995-12279-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.253.194:0
--local_ip_for_outbound_sockets=127.11.253.194
--webserver_interface=127.11.253.194
--webserver_port=0
--tserver_master_addrs=127.11.253.254:40031
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.194.keytab
--principal=kudu/127.11.253.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.253.212:34987
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785988004637995-12279-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785988004637995-12279-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785988004637995-12279-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:47:05.747404 17593 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:47:05.747632 17593 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:47:05.747656 17593 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:47:05.748773 17593 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:47:05.748809 17593 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:47:05.748835 17593 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.253.194
I20260806 03:47:05.750337 17593 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.253.212:34987
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785988004637995-12279-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785988004637995-12279-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.194.keytab
--principal=kudu/127.11.253.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.253.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785988004637995-12279-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.253.194
--webserver_port=0
--tserver_master_addrs=127.11.253.254:40031
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17593
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.253.194
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785988004637995-12279-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:47:05.750532 17593 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:47:05.750738 17593 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:47:05.756122 17603 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:47:05.756114 17605 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:47:05.756114 17601 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:47:05.756559 17593 server_base.cc:1034] running on GCE node
I20260806 03:47:05.756779 17593 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:47:05.757035 17593 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:47:05.758204 17593 hybrid_clock.cc:648] HybridClock initialized: now 1785988025758200 us; error 41 us; skew 500 ppm
Aug 06 03:47:05 dist-test-slave-6ntk krb5kdc[17226](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988025, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.253.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:47:05.760983 17593 init.cc:377] Logged in from keytab as kudu/127.11.253.194@KRBTEST.COM (short username kudu)
I20260806 03:47:05.761499 17593 webserver.cc:460] Webserver started at http://127.11.253.194:36859/ using document root <none> and password file <none>
I20260806 03:47:05.761739 17593 fs_manager.cc:359] Metadata directory not provided
I20260806 03:47:05.761808 17593 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:47:05.761997 17593 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:47:05.763096 17593 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785988004637995-12279-0/minicluster-data/ts-1/data/instance:
uuid: "2d7514cc8d634576a28289ca17482a1f"
format_stamp: "Formatted at 2026-08-06 03:47:05 on dist-test-slave-6ntk"
server_key: "ce9d29628756141555a88e77e5bdc61f"
server_key_iv: "9a90ea4a70054b8b43eba4696d3a6e35"
server_key_version: "encryptionkey@0"
I20260806 03:47:05.763585 17593 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785988004637995-12279-0/minicluster-data/ts-1/wal/instance:
uuid: "2d7514cc8d634576a28289ca17482a1f"
format_stamp: "Formatted at 2026-08-06 03:47:05 on dist-test-slave-6ntk"
server_key: "ce9d29628756141555a88e77e5bdc61f"
server_key_iv: "9a90ea4a70054b8b43eba4696d3a6e35"
server_key_version: "encryptionkey@0"
I20260806 03:47:05.765192 17593 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 03:47:05.766158 17616 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:47:05.766376 17593 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 03:47:05.766490 17593 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785988004637995-12279-0/minicluster-data/ts-1/data,/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785988004637995-12279-0/minicluster-data/ts-1/wal
uuid: "2d7514cc8d634576a28289ca17482a1f"
format_stamp: "Formatted at 2026-08-06 03:47:05 on dist-test-slave-6ntk"
server_key: "ce9d29628756141555a88e77e5bdc61f"
server_key_iv: "9a90ea4a70054b8b43eba4696d3a6e35"
server_key_version: "encryptionkey@0"
I20260806 03:47:05.766567 17593 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785988004637995-12279-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785988004637995-12279-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785988004637995-12279-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:47:05.777462 17593 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:47:05.779750 17593 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:47:05.779914 17593 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:47:05.780150 17593 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 03:47:05.780471 17593 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 03:47:05.780530 17593 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 03:47:05.780575 17593 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 03:47:05.780602 17593 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 03:47:05.787118 17593 rpc_server.cc:307] RPC server started. Bound to: 127.11.253.194:41537
I20260806 03:47:05.787199 17788 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.253.194:41537 every 8 connection(s)
I20260806 03:47:05.787531 17593 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785988004637995-12279-0/minicluster-data/ts-1/data/info.pb
Aug 06 03:47:05 dist-test-slave-6ntk krb5kdc[17226](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988025, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.253.194@KRBTEST.COM for kudu/127.11.253.254@KRBTEST.COM
I20260806 03:47:05.797003 17353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:05.788326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.253.194:46973 (local address 127.11.253.254:40031)
0806 03:47:05.788405 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0806 03:47:05.788407 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:47:05.788625 (+ 218us) server_negotiation.cc:407] Connection header received
0806 03:47:05.789561 (+ 936us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:47:05.789562 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:47:05.789576 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:47:05.789639 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:47:05.790345 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:05.790859 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:05.791864 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:05.792058 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:05.792125 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:05.794278 (+ 2153us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:47:05.794280 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:47:05.794280 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:47:05.794287 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:47:05.795754 (+ 1467us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:47:05.796162 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:47:05.796162 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:47:05.796163 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:47:05.796216 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:47:05.796440 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:47:05.796440 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:47:05.796440 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:47:05.796542 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:47:05.796623 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0806 03:47:05.796868 (+ 245us) server_negotiation.cc:299] Negotiation successful
0806 03:47:05.796933 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260806 03:47:05.797246 17793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:05.788319 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:40031 (local address 127.11.253.194:46973)
0806 03:47:05.788490 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0806 03:47:05.788526 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0806 03:47:05.789449 (+ 923us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:47:05.789715 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:47:05.789718 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:47:05.790062 (+ 344us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:47:05.790254 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:05.790258 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:05.790949 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:05.790950 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:05.791751 (+ 801us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:05.791753 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:05.792142 (+ 389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:05.792142 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:05.792244 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:05.792750 (+ 506us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:47:05.792758 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:47:05.794164 (+ 1406us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:47:05.795838 (+ 1674us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:47:05.795846 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:47:05.795848 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:47:05.796089 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:47:05.796275 (+ 186us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:47:05.796276 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:47:05.796276 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:47:05.796394 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:47:05.796623 (+ 229us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:47:05.796628 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:47:05.796746 (+ 118us) client_negotiation.cc:769] Sending connection context
0806 03:47:05.796834 (+ 88us) client_negotiation.cc:240] Negotiation successful
0806 03:47:05.796976 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":105,"threads_started":1}
I20260806 03:47:05.797307 12279 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskToVWRe/build/release/bin/kudu as pid 17593
I20260806 03:47:05.797475 12279 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785988004637995-12279-0/minicluster-data/ts-1/wal/instance
I20260806 03:47:05.797698 12279 external_mini_cluster.cc:1428] Setting key e4b70348ad7c3e3f7f82a45dcf97ec35
I20260806 03:47:05.797858 17789 heartbeater.cc:347] Connected to a master server at 127.11.253.254:40031
I20260806 03:47:05.797991 17789 heartbeater.cc:464] Registering TS with master...
I20260806 03:47:05.798187 17789 heartbeater.cc:511] Master 127.11.253.254:40031 requested a full tablet report, sending...
I20260806 03:47:05.798575 17293 ts_manager.cc:194] Registered new tserver with Master: 2d7514cc8d634576a28289ca17482a1f (127.11.253.194:41537)
I20260806 03:47:05.798967 17293 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.253.194@KRBTEST.COM'} at 127.11.253.194:46973
WARNING: no policy specified for kudu/127.11.253.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.253.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.253.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.195.keytab.
Entry for principal kudu/127.11.253.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.195.keytab.
I20260806 03:47:05.825515 12279 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskToVWRe/build/release/bin/kudu
/tmp/dist-test-taskToVWRe/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785988004637995-12279-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785988004637995-12279-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785988004637995-12279-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785988004637995-12279-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.253.195:0
--local_ip_for_outbound_sockets=127.11.253.195
--webserver_interface=127.11.253.195
--webserver_port=0
--tserver_master_addrs=127.11.253.254:40031
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.195.keytab
--principal=kudu/127.11.253.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.253.212:34987
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785988004637995-12279-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785988004637995-12279-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785988004637995-12279-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:47:05.910768 17797 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:47:05.911091 17797 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:47:05.911123 17797 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:47:05.912354 17797 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:47:05.912395 17797 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:47:05.912421 17797 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.253.195
I20260806 03:47:05.914167 17797 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.253.212:34987
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785988004637995-12279-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785988004637995-12279-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.195.keytab
--principal=kudu/127.11.253.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.253.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785988004637995-12279-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.253.195
--webserver_port=0
--tserver_master_addrs=127.11.253.254:40031
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17797
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.253.195
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785988004637995-12279-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:47:05.914390 17797 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:47:05.914577 17797 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:47:05.920082 17808 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:47:05.920125 17804 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:47:05.920125 17806 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:47:05.920230 17797 server_base.cc:1034] running on GCE node
I20260806 03:47:05.920557 17797 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:47:05.920773 17797 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:47:05.921993 17797 hybrid_clock.cc:648] HybridClock initialized: now 1785988025921964 us; error 44 us; skew 500 ppm
Aug 06 03:47:05 dist-test-slave-6ntk krb5kdc[17226](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988025, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.253.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:47:05.924660 17797 init.cc:377] Logged in from keytab as kudu/127.11.253.195@KRBTEST.COM (short username kudu)
I20260806 03:47:05.925266 17797 webserver.cc:460] Webserver started at http://127.11.253.195:33147/ using document root <none> and password file <none>
I20260806 03:47:05.925504 17797 fs_manager.cc:359] Metadata directory not provided
I20260806 03:47:05.925552 17797 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:47:05.925715 17797 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:47:05.926956 17797 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785988004637995-12279-0/minicluster-data/ts-2/data/instance:
uuid: "62edd8d7d2534f14a7b1990d116b21eb"
format_stamp: "Formatted at 2026-08-06 03:47:05 on dist-test-slave-6ntk"
server_key: "6d5f993493a1fa4d4ce77a016bc6e9b7"
server_key_iv: "777e58af00840f33c9534a509b3de5d7"
server_key_version: "encryptionkey@0"
I20260806 03:47:05.927380 17797 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785988004637995-12279-0/minicluster-data/ts-2/wal/instance:
uuid: "62edd8d7d2534f14a7b1990d116b21eb"
format_stamp: "Formatted at 2026-08-06 03:47:05 on dist-test-slave-6ntk"
server_key: "6d5f993493a1fa4d4ce77a016bc6e9b7"
server_key_iv: "777e58af00840f33c9534a509b3de5d7"
server_key_version: "encryptionkey@0"
I20260806 03:47:05.928774 17797 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 03:47:05.929571 17818 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:47:05.929785 17797 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 03:47:05.929911 17797 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785988004637995-12279-0/minicluster-data/ts-2/data,/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785988004637995-12279-0/minicluster-data/ts-2/wal
uuid: "62edd8d7d2534f14a7b1990d116b21eb"
format_stamp: "Formatted at 2026-08-06 03:47:05 on dist-test-slave-6ntk"
server_key: "6d5f993493a1fa4d4ce77a016bc6e9b7"
server_key_iv: "777e58af00840f33c9534a509b3de5d7"
server_key_version: "encryptionkey@0"
I20260806 03:47:05.929991 17797 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785988004637995-12279-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785988004637995-12279-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785988004637995-12279-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:47:05.954844 17797 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:47:05.956889 17797 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:47:05.957055 17797 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:47:05.957317 17797 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 03:47:05.957664 17797 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 03:47:05.957731 17797 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 03:47:05.957774 17797 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 03:47:05.957801 17797 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 03:47:05.963879 17797 rpc_server.cc:307] RPC server started. Bound to: 127.11.253.195:33801
I20260806 03:47:05.963943 17986 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.253.195:33801 every 8 connection(s)
I20260806 03:47:05.964267 17797 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785988004637995-12279-0/minicluster-data/ts-2/data/info.pb
Aug 06 03:47:05 dist-test-slave-6ntk krb5kdc[17226](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988025, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.253.195@KRBTEST.COM for kudu/127.11.253.254@KRBTEST.COM
I20260806 03:47:05.971917 12279 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskToVWRe/build/release/bin/kudu as pid 17797
I20260806 03:47:05.972051 12279 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785988004637995-12279-0/minicluster-data/ts-2/wal/instance
I20260806 03:47:05.972270 12279 external_mini_cluster.cc:1428] Setting key 4775b31eb98bd06766cd502b41ecc39d
I20260806 03:47:05.974050 17353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:05.965047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.253.195:57675 (local address 127.11.253.254:40031)
0806 03:47:05.965112 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0806 03:47:05.965114 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:47:05.965317 (+ 203us) server_negotiation.cc:407] Connection header received
0806 03:47:05.966262 (+ 945us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:47:05.966263 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:47:05.966274 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:47:05.966347 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:47:05.966922 (+ 575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:05.967411 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:05.968377 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:05.968560 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:05.968610 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:05.970973 (+ 2363us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:47:05.970974 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:47:05.970974 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:47:05.970981 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:47:05.972599 (+ 1618us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:47:05.973120 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:47:05.973120 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:47:05.973121 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:47:05.973184 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:47:05.973473 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:47:05.973474 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:47:05.973474 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:47:05.973584 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:47:05.973642 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0806 03:47:05.973923 (+ 281us) server_negotiation.cc:299] Negotiation successful
0806 03:47:05.973987 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260806 03:47:05.974270 17989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:05.965003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:40031 (local address 127.11.253.195:57675)
0806 03:47:05.965178 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0806 03:47:05.965220 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 03:47:05.966172 (+ 952us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:47:05.966402 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:47:05.966406 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:47:05.966653 (+ 247us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:47:05.966852 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:05.966856 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:05.967495 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:05.967496 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:05.968305 (+ 809us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:05.968307 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:05.968649 (+ 342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:05.968650 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:05.968781 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:05.969374 (+ 593us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:47:05.969382 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:47:05.970882 (+ 1500us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:47:05.972694 (+ 1812us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:47:05.972700 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:47:05.972703 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:47:05.973039 (+ 336us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:47:05.973273 (+ 234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:47:05.973274 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:47:05.973274 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:47:05.973413 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:47:05.973634 (+ 221us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:47:05.973637 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:47:05.973757 (+ 120us) client_negotiation.cc:769] Sending connection context
0806 03:47:05.973833 (+ 76us) client_negotiation.cc:240] Negotiation successful
0806 03:47:05.974021 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":94,"threads_started":1}
I20260806 03:47:05.974792 17987 heartbeater.cc:347] Connected to a master server at 127.11.253.254:40031
I20260806 03:47:05.974900 17987 heartbeater.cc:464] Registering TS with master...
I20260806 03:47:05.975075 17987 heartbeater.cc:511] Master 127.11.253.254:40031 requested a full tablet report, sending...
I20260806 03:47:05.975423 17293 ts_manager.cc:194] Registered new tserver with Master: 62edd8d7d2534f14a7b1990d116b21eb (127.11.253.195:33801)
I20260806 03:47:05.975773 17293 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.253.195@KRBTEST.COM'} at 127.11.253.195:57675
I20260806 03:47:05.984499 12279 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 06 03:47:05 dist-test-slave-6ntk krb5kdc[17226](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988025, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 03:47:06 dist-test-slave-6ntk krb5kdc[17226](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988025, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.253.193@KRBTEST.COM
I20260806 03:47:06.010959 17999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:06.002395 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48308 (local address 127.11.253.193:46007)
0806 03:47:06.002594 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0806 03:47:06.002599 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 03:47:06.002631 (+ 32us) server_negotiation.cc:407] Connection header received
0806 03:47:06.002658 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:47:06.002660 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:47:06.002675 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:47:06.002732 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:47:06.003353 (+ 621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:06.004068 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:06.004835 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:06.005156 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:06.005210 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:06.007761 (+ 2551us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:47:06.007771 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:47:06.007771 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:47:06.007779 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:47:06.009562 (+ 1783us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:47:06.010015 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:47:06.010018 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:47:06.010019 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:47:06.010083 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:47:06.010378 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:47:06.010380 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:47:06.010380 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:47:06.010562 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:47:06.010736 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0806 03:47:06.010792 (+ 56us) server_negotiation.cc:299] Negotiation successful
0806 03:47:06.010868 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":164,"thread_start_us":111,"threads_started":1}
Aug 06 03:47:06 dist-test-slave-6ntk krb5kdc[17226](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988025, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.253.254@KRBTEST.COM
I20260806 03:47:06.019908 17353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:06.012720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40406 (local address 127.11.253.254:40031)
0806 03:47:06.012805 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0806 03:47:06.012807 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:47:06.012816 (+ 9us) server_negotiation.cc:407] Connection header received
0806 03:47:06.012911 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:47:06.012912 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:47:06.012923 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:47:06.012982 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:47:06.013478 (+ 496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:06.014078 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:06.014819 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:06.015018 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:06.015097 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:06.017351 (+ 2254us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:47:06.017352 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:47:06.017353 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:47:06.017360 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:47:06.018679 (+ 1319us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:47:06.019057 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:47:06.019059 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:47:06.019059 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:47:06.019120 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:47:06.019337 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:47:06.019338 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:47:06.019339 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:47:06.019497 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:47:06.019610 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0806 03:47:06.019707 (+ 97us) server_negotiation.cc:299] Negotiation successful
0806 03:47:06.019794 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
W20260806 03:47:06.020188 17293 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40406
I20260806 03:47:06.020591 12279 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskToVWRe/build/release/bin/kudu with pid 17389
I20260806 03:47:06.037147 12279 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskToVWRe/build/release/bin/kudu with pid 17593
I20260806 03:47:06.045977 12279 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskToVWRe/build/release/bin/kudu with pid 17797
I20260806 03:47:06.058138 12279 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskToVWRe/build/release/bin/kudu with pid 17242
2026-08-06T03:47:06Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (2163 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785988004637995-12279-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 03:47:06 dist-test-slave-6ntk krb5kdc[18008](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 03:47:06 dist-test-slave-6ntk krb5kdc[18008](info): set up 2 sockets
Aug 06 03:47:06 dist-test-slave-6ntk krb5kdc[18008](info): commencing operation
krb5kdc: starting...
W20260806 03:47:07.288651 12279 mini_kdc.cc:121] Time spent starting KDC: real 1.204s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785988004637995-12279-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785988004637995-12279-0/krb5kdc/test-user.keytab.
Aug 06 03:47:07 dist-test-slave-6ntk krb5kdc[18008](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988027, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T03:47:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T03:47:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.253.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.253.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.253.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.254.keytab.
Entry for principal kudu/127.11.253.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.254.keytab.
I20260806 03:47:07.418291 12279 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskToVWRe/build/release/bin/kudu
/tmp/dist-test-taskToVWRe/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785988004637995-12279-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785988004637995-12279-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785988004637995-12279-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785988004637995-12279-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.253.254:46651
--webserver_interface=127.11.253.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.254.keytab
--principal=kudu/127.11.253.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.253.212:35947
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.253.254:46651
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785988004637995-12279-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785988004637995-12279-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785988004637995-12279-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:47:07.503572 18028 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:47:07.503772 18028 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:47:07.503793 18028 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:47:07.504912 18028 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 03:47:07.504952 18028 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:47:07.504966 18028 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:47:07.504979 18028 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 03:47:07.505024 18028 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 03:47:07.506620 18028 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.253.212:35947
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785988004637995-12279-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785988004637995-12279-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.253.254:46651
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.254.keytab
--principal=kudu/127.11.253.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.253.254:46651
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785988004637995-12279-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.253.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18028
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785988004637995-12279-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:47:07.506879 18028 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:47:07.507123 18028 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:47:07.512409 18039 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:47:07.512416 18041 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:47:07.512625 18028 server_base.cc:1034] running on GCE node
W20260806 03:47:07.512416 18044 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:47:07.512943 18028 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:47:07.513238 18028 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:47:07.514413 18028 hybrid_clock.cc:648] HybridClock initialized: now 1785988027514415 us; error 45 us; skew 500 ppm
Aug 06 03:47:07 dist-test-slave-6ntk krb5kdc[18008](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988027, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.253.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:47:07.521626 18028 init.cc:377] Logged in from keytab as kudu/127.11.253.254@KRBTEST.COM (short username kudu)
I20260806 03:47:07.522024 18028 fs_manager.cc:359] Metadata directory not provided
I20260806 03:47:07.522078 18028 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:47:07.522176 18028 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:47:07.523216 18028 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785988004637995-12279-0/minicluster-data/master-0/data/instance:
uuid: "70ab0d51fcb04d96ba325ddb1b919ffd"
format_stamp: "Formatted at 2026-08-06 03:47:07 on dist-test-slave-6ntk"
server_key: "30c2a39bfb320b85157447734893e585"
server_key_iv: "53dc5823bde8f0041c9448c511725f93"
server_key_version: "encryptionkey@0"
I20260806 03:47:07.523609 18028 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785988004637995-12279-0/minicluster-data/master-0/wal/instance:
uuid: "70ab0d51fcb04d96ba325ddb1b919ffd"
format_stamp: "Formatted at 2026-08-06 03:47:07 on dist-test-slave-6ntk"
server_key: "30c2a39bfb320b85157447734893e585"
server_key_iv: "53dc5823bde8f0041c9448c511725f93"
server_key_version: "encryptionkey@0"
I20260806 03:47:07.524932 18028 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 03:47:07.525715 18052 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:47:07.525923 18028 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 03:47:07.526024 18028 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785988004637995-12279-0/minicluster-data/master-0/data,/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785988004637995-12279-0/minicluster-data/master-0/wal
uuid: "70ab0d51fcb04d96ba325ddb1b919ffd"
format_stamp: "Formatted at 2026-08-06 03:47:07 on dist-test-slave-6ntk"
server_key: "30c2a39bfb320b85157447734893e585"
server_key_iv: "53dc5823bde8f0041c9448c511725f93"
server_key_version: "encryptionkey@0"
I20260806 03:47:07.526083 18028 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785988004637995-12279-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785988004637995-12279-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785988004637995-12279-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:47:07.543802 18028 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:47:07.546247 18028 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:47:07.546367 18028 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:47:07.550534 18028 rpc_server.cc:307] RPC server started. Bound to: 127.11.253.254:46651
I20260806 03:47:07.550598 18133 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.253.254:46651 every 8 connection(s)
I20260806 03:47:07.550904 18028 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785988004637995-12279-0/minicluster-data/master-0/data/info.pb
I20260806 03:47:07.552042 18134 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:47:07.554080 12279 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskToVWRe/build/release/bin/kudu as pid 18028
I20260806 03:47:07.554162 12279 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785988004637995-12279-0/minicluster-data/master-0/wal/instance
I20260806 03:47:07.554342 12279 external_mini_cluster.cc:1428] Setting key 1ae889b1d11821af3f5e6d5962b9cfaf
I20260806 03:47:07.554567 18134 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 70ab0d51fcb04d96ba325ddb1b919ffd: Bootstrap starting.
I20260806 03:47:07.555425 18134 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 70ab0d51fcb04d96ba325ddb1b919ffd: Neither blocks nor log segments found. Creating new log.
I20260806 03:47:07.555768 18134 log.cc:824] T 00000000000000000000000000000000 P 70ab0d51fcb04d96ba325ddb1b919ffd: Log is configured to *not* fsync() on all Append() calls
I20260806 03:47:07.556576 18134 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 70ab0d51fcb04d96ba325ddb1b919ffd: No bootstrap required, opened a new log
I20260806 03:47:07.558486 18134 raft_consensus.cc:348] T 00000000000000000000000000000000 P 70ab0d51fcb04d96ba325ddb1b919ffd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70ab0d51fcb04d96ba325ddb1b919ffd" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 46651 } }
I20260806 03:47:07.558643 18134 raft_consensus.cc:374] T 00000000000000000000000000000000 P 70ab0d51fcb04d96ba325ddb1b919ffd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:47:07.558696 18134 raft_consensus.cc:729] T 00000000000000000000000000000000 P 70ab0d51fcb04d96ba325ddb1b919ffd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 70ab0d51fcb04d96ba325ddb1b919ffd, State: Initialized, Role: FOLLOWER
I20260806 03:47:07.558861 18134 consensus_queue.cc:260] T 00000000000000000000000000000000 P 70ab0d51fcb04d96ba325ddb1b919ffd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70ab0d51fcb04d96ba325ddb1b919ffd" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 46651 } }
I20260806 03:47:07.558954 18134 raft_consensus.cc:388] T 00000000000000000000000000000000 P 70ab0d51fcb04d96ba325ddb1b919ffd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 03:47:07.558995 18134 raft_consensus.cc:482] T 00000000000000000000000000000000 P 70ab0d51fcb04d96ba325ddb1b919ffd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 03:47:07.559047 18134 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 70ab0d51fcb04d96ba325ddb1b919ffd [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:47:07.559621 18134 raft_consensus.cc:504] T 00000000000000000000000000000000 P 70ab0d51fcb04d96ba325ddb1b919ffd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70ab0d51fcb04d96ba325ddb1b919ffd" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 46651 } }
I20260806 03:47:07.559743 18134 leader_election.cc:302] T 00000000000000000000000000000000 P 70ab0d51fcb04d96ba325ddb1b919ffd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 70ab0d51fcb04d96ba325ddb1b919ffd; no voters:
I20260806 03:47:07.559934 18134 leader_election.cc:288] T 00000000000000000000000000000000 P 70ab0d51fcb04d96ba325ddb1b919ffd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 03:47:07.559966 18142 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 70ab0d51fcb04d96ba325ddb1b919ffd [term 1 FOLLOWER]: Leader election won for term 1
I20260806 03:47:07.560091 18142 raft_consensus.cc:686] T 00000000000000000000000000000000 P 70ab0d51fcb04d96ba325ddb1b919ffd [term 1 LEADER]: Becoming Leader. State: Replica: 70ab0d51fcb04d96ba325ddb1b919ffd, State: Running, Role: LEADER
I20260806 03:47:07.560174 18142 consensus_queue.cc:237] T 00000000000000000000000000000000 P 70ab0d51fcb04d96ba325ddb1b919ffd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70ab0d51fcb04d96ba325ddb1b919ffd" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 46651 } }
I20260806 03:47:07.560204 18134 sys_catalog.cc:564] T 00000000000000000000000000000000 P 70ab0d51fcb04d96ba325ddb1b919ffd [sys.catalog]: configured and running, proceeding with master startup.
I20260806 03:47:07.560591 18142 sys_catalog.cc:455] T 00000000000000000000000000000000 P 70ab0d51fcb04d96ba325ddb1b919ffd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 70ab0d51fcb04d96ba325ddb1b919ffd. Latest consensus state: current_term: 1 leader_uuid: "70ab0d51fcb04d96ba325ddb1b919ffd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70ab0d51fcb04d96ba325ddb1b919ffd" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 46651 } } }
I20260806 03:47:07.560601 18143 sys_catalog.cc:455] T 00000000000000000000000000000000 P 70ab0d51fcb04d96ba325ddb1b919ffd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "70ab0d51fcb04d96ba325ddb1b919ffd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70ab0d51fcb04d96ba325ddb1b919ffd" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 46651 } } }
I20260806 03:47:07.560879 18143 sys_catalog.cc:458] T 00000000000000000000000000000000 P 70ab0d51fcb04d96ba325ddb1b919ffd [sys.catalog]: This master's current role is: LEADER
I20260806 03:47:07.560870 18142 sys_catalog.cc:458] T 00000000000000000000000000000000 P 70ab0d51fcb04d96ba325ddb1b919ffd [sys.catalog]: This master's current role is: LEADER
Aug 06 03:47:07 dist-test-slave-6ntk krb5kdc[18008](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988027, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.253.254@KRBTEST.COM
W20260806 03:47:07.562630 18164 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 70ab0d51fcb04d96ba325ddb1b919ffd: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 03:47:07.564894 18164 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 03:47:07.564970 18165 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 03:47:07.565266 18165 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 03:47:07.567294 18137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:07.554907 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54800 (local address 127.11.253.254:46651)
0806 03:47:07.555115 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0806 03:47:07.555125 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 03:47:07.555161 (+ 36us) server_negotiation.cc:407] Connection header received
0806 03:47:07.555860 (+ 699us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:47:07.555869 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:47:07.555933 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:47:07.556081 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:47:07.556777 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:07.557677 (+ 900us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:07.558479 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:07.559005 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:07.559154 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:07.562100 (+ 2946us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:47:07.562105 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:47:07.562105 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:47:07.562119 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:47:07.564353 (+ 2234us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:47:07.564808 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:47:07.564812 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:47:07.564812 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:47:07.565031 (+ 219us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:47:07.565443 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:47:07.565444 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:47:07.565445 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:47:07.565751 (+ 306us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:47:07.566793 (+ 1042us) server_negotiation.cc:1035] Waiting for connection context
0806 03:47:07.566868 (+ 75us) server_negotiation.cc:299] Negotiation successful
0806 03:47:07.567034 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":100,"threads_started":1}
I20260806 03:47:07.567870 18165 catalog_manager.cc:1347] Generated new cluster ID: c9539239239b43c6bc732beac864df4b
I20260806 03:47:07.567970 18165 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 03:47:07.581149 18165 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 03:47:07.581874 18165 catalog_manager.cc:1504] Loading token signing keys...
I20260806 03:47:07.598307 18165 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 70ab0d51fcb04d96ba325ddb1b919ffd: Generated new TSK 0
I20260806 03:47:07.598541 18165 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.253.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.253.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.253.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.193.keytab.
Entry for principal kudu/127.11.253.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.193.keytab.
I20260806 03:47:07.640353 12279 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskToVWRe/build/release/bin/kudu
/tmp/dist-test-taskToVWRe/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785988004637995-12279-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785988004637995-12279-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785988004637995-12279-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785988004637995-12279-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.253.193:0
--local_ip_for_outbound_sockets=127.11.253.193
--webserver_interface=127.11.253.193
--webserver_port=0
--tserver_master_addrs=127.11.253.254:46651
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.193.keytab
--principal=kudu/127.11.253.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.253.212:35947
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785988004637995-12279-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785988004637995-12279-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785988004637995-12279-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:47:07.741540 18172 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:47:07.741740 18172 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:47:07.741765 18172 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:47:07.742862 18172 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:47:07.742898 18172 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:47:07.742923 18172 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.253.193
I20260806 03:47:07.744365 18172 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.253.212:35947
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785988004637995-12279-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785988004637995-12279-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.193.keytab
--principal=kudu/127.11.253.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.253.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785988004637995-12279-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.253.193
--webserver_port=0
--tserver_master_addrs=127.11.253.254:46651
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18172
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.253.193
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785988004637995-12279-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:47:07.744553 18172 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:47:07.744729 18172 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:47:07.749791 18179 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:47:07.749804 18178 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:47:07.749776 18183 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:47:07.750011 18172 server_base.cc:1034] running on GCE node
I20260806 03:47:07.750201 18172 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:47:07.750439 18172 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:47:07.751575 18172 hybrid_clock.cc:648] HybridClock initialized: now 1785988027751563 us; error 29 us; skew 500 ppm
Aug 06 03:47:07 dist-test-slave-6ntk krb5kdc[18008](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988027, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.253.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:47:07.754150 18172 init.cc:377] Logged in from keytab as kudu/127.11.253.193@KRBTEST.COM (short username kudu)
I20260806 03:47:07.754456 18172 fs_manager.cc:359] Metadata directory not provided
I20260806 03:47:07.754500 18172 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:47:07.754588 18172 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:47:07.755620 18172 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785988004637995-12279-0/minicluster-data/ts-0/data/instance:
uuid: "c27d8d93b40549e29a6bac8bd57d24ce"
format_stamp: "Formatted at 2026-08-06 03:47:07 on dist-test-slave-6ntk"
server_key: "e30619a90dfc05d919fdb8c2811f771c"
server_key_iv: "4e03cbed69e50f1189bec3fc50fd5c0e"
server_key_version: "encryptionkey@0"
I20260806 03:47:07.755923 18172 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785988004637995-12279-0/minicluster-data/ts-0/wal/instance:
uuid: "c27d8d93b40549e29a6bac8bd57d24ce"
format_stamp: "Formatted at 2026-08-06 03:47:07 on dist-test-slave-6ntk"
server_key: "e30619a90dfc05d919fdb8c2811f771c"
server_key_iv: "4e03cbed69e50f1189bec3fc50fd5c0e"
server_key_version: "encryptionkey@0"
I20260806 03:47:07.757141 18172 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260806 03:47:07.757918 18193 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:47:07.758112 18172 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260806 03:47:07.758203 18172 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785988004637995-12279-0/minicluster-data/ts-0/data,/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785988004637995-12279-0/minicluster-data/ts-0/wal
uuid: "c27d8d93b40549e29a6bac8bd57d24ce"
format_stamp: "Formatted at 2026-08-06 03:47:07 on dist-test-slave-6ntk"
server_key: "e30619a90dfc05d919fdb8c2811f771c"
server_key_iv: "4e03cbed69e50f1189bec3fc50fd5c0e"
server_key_version: "encryptionkey@0"
I20260806 03:47:07.758283 18172 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785988004637995-12279-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785988004637995-12279-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785988004637995-12279-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:47:07.788618 18172 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:47:07.790755 18172 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:47:07.790953 18172 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:47:07.791225 18172 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 03:47:07.791632 18172 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 03:47:07.791667 18172 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 03:47:07.791692 18172 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 03:47:07.791739 18172 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 03:47:07.797973 18172 rpc_server.cc:307] RPC server started. Bound to: 127.11.253.193:33249
I20260806 03:47:07.798032 18365 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.253.193:33249 every 8 connection(s)
I20260806 03:47:07.798334 18172 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785988004637995-12279-0/minicluster-data/ts-0/data/info.pb
Aug 06 03:47:07 dist-test-slave-6ntk krb5kdc[18008](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988027, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.253.193@KRBTEST.COM for kudu/127.11.253.254@KRBTEST.COM
I20260806 03:47:07.807153 12279 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskToVWRe/build/release/bin/kudu as pid 18172
I20260806 03:47:07.807245 12279 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785988004637995-12279-0/minicluster-data/ts-0/wal/instance
I20260806 03:47:07.807480 12279 external_mini_cluster.cc:1428] Setting key c92c338327d62ff333d792e8ab355d36
I20260806 03:47:07.807819 18137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:07.799064 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.253.193:40029 (local address 127.11.253.254:46651)
0806 03:47:07.799146 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0806 03:47:07.799148 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:47:07.799327 (+ 179us) server_negotiation.cc:407] Connection header received
0806 03:47:07.799797 (+ 470us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:47:07.799798 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:47:07.799808 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:47:07.799863 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:47:07.800838 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:07.801350 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:07.802350 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:07.802553 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:07.802618 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:07.804856 (+ 2238us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:47:07.804858 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:47:07.804858 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:47:07.804866 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:47:07.806321 (+ 1455us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:47:07.806723 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:47:07.806724 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:47:07.806725 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:47:07.806787 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:47:07.807045 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:47:07.807046 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:47:07.807046 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:47:07.807175 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:47:07.807258 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0806 03:47:07.807613 (+ 355us) server_negotiation.cc:299] Negotiation successful
0806 03:47:07.807713 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260806 03:47:07.807996 18370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:07.799017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:46651 (local address 127.11.253.193:40029)
0806 03:47:07.799207 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0806 03:47:07.799245 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0806 03:47:07.799717 (+ 472us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:47:07.799920 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:47:07.799923 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:47:07.800201 (+ 278us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:47:07.800744 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:07.800750 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:07.801434 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:07.801434 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:07.802249 (+ 815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:07.802251 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:07.802635 (+ 384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:07.802635 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:07.802734 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:07.803289 (+ 555us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:47:07.803298 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:47:07.804756 (+ 1458us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:47:07.806407 (+ 1651us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:47:07.806416 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:47:07.806418 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:47:07.806644 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:47:07.806852 (+ 208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:47:07.806853 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:47:07.806853 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:47:07.806961 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:47:07.807285 (+ 324us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:47:07.807293 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:47:07.807444 (+ 151us) client_negotiation.cc:769] Sending connection context
0806 03:47:07.807565 (+ 121us) client_negotiation.cc:240] Negotiation successful
0806 03:47:07.807747 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":100,"threads_started":1}
I20260806 03:47:07.808499 18366 heartbeater.cc:347] Connected to a master server at 127.11.253.254:46651
I20260806 03:47:07.808614 18366 heartbeater.cc:464] Registering TS with master...
I20260806 03:47:07.808782 18366 heartbeater.cc:511] Master 127.11.253.254:46651 requested a full tablet report, sending...
I20260806 03:47:07.809312 18075 ts_manager.cc:194] Registered new tserver with Master: c27d8d93b40549e29a6bac8bd57d24ce (127.11.253.193:33249)
I20260806 03:47:07.810026 18075 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.253.193@KRBTEST.COM'} at 127.11.253.193:40029
WARNING: no policy specified for kudu/127.11.253.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.253.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.253.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.194.keytab.
Entry for principal kudu/127.11.253.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.194.keytab.
I20260806 03:47:07.835094 12279 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskToVWRe/build/release/bin/kudu
/tmp/dist-test-taskToVWRe/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785988004637995-12279-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785988004637995-12279-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785988004637995-12279-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785988004637995-12279-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.253.194:0
--local_ip_for_outbound_sockets=127.11.253.194
--webserver_interface=127.11.253.194
--webserver_port=0
--tserver_master_addrs=127.11.253.254:46651
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.194.keytab
--principal=kudu/127.11.253.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.253.212:35947
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785988004637995-12279-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785988004637995-12279-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785988004637995-12279-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:47:07.919214 18377 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:47:07.919486 18377 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:47:07.919525 18377 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:47:07.920665 18377 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:47:07.920734 18377 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:47:07.920778 18377 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.253.194
I20260806 03:47:07.922315 18377 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.253.212:35947
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785988004637995-12279-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785988004637995-12279-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.194.keytab
--principal=kudu/127.11.253.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.253.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785988004637995-12279-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.253.194
--webserver_port=0
--tserver_master_addrs=127.11.253.254:46651
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18377
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.253.194
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785988004637995-12279-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:47:07.922593 18377 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:47:07.922828 18377 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:47:07.928021 18393 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:47:07.928025 18395 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:47:07.928015 18391 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:47:07.928314 18377 server_base.cc:1034] running on GCE node
I20260806 03:47:07.928524 18377 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:47:07.928782 18377 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:47:07.929924 18377 hybrid_clock.cc:648] HybridClock initialized: now 1785988027929908 us; error 29 us; skew 500 ppm
Aug 06 03:47:07 dist-test-slave-6ntk krb5kdc[18008](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988027, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.253.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:47:07.932502 18377 init.cc:377] Logged in from keytab as kudu/127.11.253.194@KRBTEST.COM (short username kudu)
I20260806 03:47:07.932842 18377 fs_manager.cc:359] Metadata directory not provided
I20260806 03:47:07.932893 18377 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:47:07.932983 18377 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:47:07.934033 18377 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785988004637995-12279-0/minicluster-data/ts-1/data/instance:
uuid: "2a1dc086aaeb45bb9876141dc3a08d8e"
format_stamp: "Formatted at 2026-08-06 03:47:07 on dist-test-slave-6ntk"
server_key: "e14b0491fa6cea4d8ce9d34ab83aa116"
server_key_iv: "110438e30fb9ae87a62e473221a9973c"
server_key_version: "encryptionkey@0"
I20260806 03:47:07.934388 18377 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785988004637995-12279-0/minicluster-data/ts-1/wal/instance:
uuid: "2a1dc086aaeb45bb9876141dc3a08d8e"
format_stamp: "Formatted at 2026-08-06 03:47:07 on dist-test-slave-6ntk"
server_key: "e14b0491fa6cea4d8ce9d34ab83aa116"
server_key_iv: "110438e30fb9ae87a62e473221a9973c"
server_key_version: "encryptionkey@0"
I20260806 03:47:07.935597 18377 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 03:47:07.936256 18406 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:47:07.936465 18377 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 03:47:07.936568 18377 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785988004637995-12279-0/minicluster-data/ts-1/data,/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785988004637995-12279-0/minicluster-data/ts-1/wal
uuid: "2a1dc086aaeb45bb9876141dc3a08d8e"
format_stamp: "Formatted at 2026-08-06 03:47:07 on dist-test-slave-6ntk"
server_key: "e14b0491fa6cea4d8ce9d34ab83aa116"
server_key_iv: "110438e30fb9ae87a62e473221a9973c"
server_key_version: "encryptionkey@0"
I20260806 03:47:07.936640 18377 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785988004637995-12279-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785988004637995-12279-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785988004637995-12279-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:47:07.949245 18377 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:47:07.951550 18377 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:47:07.951702 18377 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:47:07.951932 18377 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 03:47:07.952237 18377 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 03:47:07.952286 18377 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 03:47:07.952329 18377 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 03:47:07.952378 18377 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 03:47:07.958477 18377 rpc_server.cc:307] RPC server started. Bound to: 127.11.253.194:42607
I20260806 03:47:07.958518 18564 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.253.194:42607 every 8 connection(s)
I20260806 03:47:07.958799 18377 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785988004637995-12279-0/minicluster-data/ts-1/data/info.pb
I20260806 03:47:07.960546 12279 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskToVWRe/build/release/bin/kudu as pid 18377
I20260806 03:47:07.960659 12279 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785988004637995-12279-0/minicluster-data/ts-1/wal/instance
I20260806 03:47:07.960872 12279 external_mini_cluster.cc:1428] Setting key cb612ebbd046c067a6c3f96092108b3c
Aug 06 03:47:07 dist-test-slave-6ntk krb5kdc[18008](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988027, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.253.194@KRBTEST.COM for kudu/127.11.253.254@KRBTEST.COM
I20260806 03:47:07.969135 18137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:07.959527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.253.194:45577 (local address 127.11.253.254:46651)
0806 03:47:07.959594 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0806 03:47:07.959596 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:47:07.959790 (+ 194us) server_negotiation.cc:407] Connection header received
0806 03:47:07.960368 (+ 578us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:47:07.960370 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:47:07.960382 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:47:07.960467 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:47:07.961743 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:07.962403 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:07.963578 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:07.963770 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:07.963823 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:07.966284 (+ 2461us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:47:07.966286 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:47:07.966286 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:47:07.966296 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:47:07.967732 (+ 1436us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:47:07.968159 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:47:07.968160 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:47:07.968160 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:47:07.968223 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:47:07.968475 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:47:07.968476 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:47:07.968476 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:47:07.968594 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:47:07.968666 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0806 03:47:07.968956 (+ 290us) server_negotiation.cc:299] Negotiation successful
0806 03:47:07.969040 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260806 03:47:07.969338 18569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:07.959512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:46651 (local address 127.11.253.194:45577)
0806 03:47:07.959672 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0806 03:47:07.959707 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0806 03:47:07.960227 (+ 520us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:47:07.960558 (+ 331us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:47:07.960561 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:47:07.961001 (+ 440us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:47:07.961652 (+ 651us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:07.961657 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:07.962494 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:07.962495 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:07.963471 (+ 976us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:07.963475 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:07.963847 (+ 372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:07.963848 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:07.963997 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:07.964521 (+ 524us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:47:07.964532 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:47:07.966183 (+ 1651us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:47:07.967834 (+ 1651us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:47:07.967841 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:47:07.967851 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:47:07.968081 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:47:07.968291 (+ 210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:47:07.968292 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:47:07.968293 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:47:07.968408 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:47:07.968667 (+ 259us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:47:07.968676 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:47:07.968815 (+ 139us) client_negotiation.cc:769] Sending connection context
0806 03:47:07.968924 (+ 109us) client_negotiation.cc:240] Negotiation successful
0806 03:47:07.969089 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":74,"threads_started":1}
I20260806 03:47:07.969808 18565 heartbeater.cc:347] Connected to a master server at 127.11.253.254:46651
I20260806 03:47:07.969954 18565 heartbeater.cc:464] Registering TS with master...
I20260806 03:47:07.970117 18565 heartbeater.cc:511] Master 127.11.253.254:46651 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.11.253.195@KRBTEST.COM; defaulting to no policy
I20260806 03:47:07.970500 18075 ts_manager.cc:194] Registered new tserver with Master: 2a1dc086aaeb45bb9876141dc3a08d8e (127.11.253.194:42607)
I20260806 03:47:07.970839 18075 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.253.194@KRBTEST.COM'} at 127.11.253.194:45577
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.253.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.253.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.195.keytab.
Entry for principal kudu/127.11.253.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.195.keytab.
I20260806 03:47:07.989411 12279 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskToVWRe/build/release/bin/kudu
/tmp/dist-test-taskToVWRe/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785988004637995-12279-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785988004637995-12279-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785988004637995-12279-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785988004637995-12279-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.253.195:0
--local_ip_for_outbound_sockets=127.11.253.195
--webserver_interface=127.11.253.195
--webserver_port=0
--tserver_master_addrs=127.11.253.254:46651
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.195.keytab
--principal=kudu/127.11.253.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.253.212:35947
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785988004637995-12279-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785988004637995-12279-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785988004637995-12279-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:47:08.075939 18574 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:47:08.076148 18574 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:47:08.076172 18574 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:47:08.077334 18574 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:47:08.077375 18574 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:47:08.077425 18574 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.253.195
I20260806 03:47:08.078981 18574 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.253.212:35947
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785988004637995-12279-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785988004637995-12279-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.195.keytab
--principal=kudu/127.11.253.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.253.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785988004637995-12279-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.253.195
--webserver_port=0
--tserver_master_addrs=127.11.253.254:46651
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18574
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.253.195
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785988004637995-12279-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:47:08.079216 18574 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:47:08.079449 18574 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:47:08.084703 18587 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:47:08.084733 18583 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:47:08.084730 18585 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:47:08.084753 18574 server_base.cc:1034] running on GCE node
I20260806 03:47:08.085095 18574 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:47:08.085282 18574 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:47:08.086434 18574 hybrid_clock.cc:648] HybridClock initialized: now 1785988028086420 us; error 30 us; skew 500 ppm
Aug 06 03:47:08 dist-test-slave-6ntk krb5kdc[18008](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988028, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.253.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:47:08.089020 18574 init.cc:377] Logged in from keytab as kudu/127.11.253.195@KRBTEST.COM (short username kudu)
I20260806 03:47:08.089365 18574 fs_manager.cc:359] Metadata directory not provided
I20260806 03:47:08.089433 18574 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:47:08.089581 18574 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:47:08.090695 18574 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785988004637995-12279-0/minicluster-data/ts-2/data/instance:
uuid: "538dd97c7dc7490484e60c94a690ebe7"
format_stamp: "Formatted at 2026-08-06 03:47:08 on dist-test-slave-6ntk"
server_key: "2e79c48ad55622e3159e9f7352ceac7f"
server_key_iv: "d7960b16fa74add839378f7ab3fc5a4f"
server_key_version: "encryptionkey@0"
I20260806 03:47:08.091077 18574 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785988004637995-12279-0/minicluster-data/ts-2/wal/instance:
uuid: "538dd97c7dc7490484e60c94a690ebe7"
format_stamp: "Formatted at 2026-08-06 03:47:08 on dist-test-slave-6ntk"
server_key: "2e79c48ad55622e3159e9f7352ceac7f"
server_key_iv: "d7960b16fa74add839378f7ab3fc5a4f"
server_key_version: "encryptionkey@0"
I20260806 03:47:08.092432 18574 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260806 03:47:08.093160 18596 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:47:08.093343 18574 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 03:47:08.093451 18574 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785988004637995-12279-0/minicluster-data/ts-2/data,/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785988004637995-12279-0/minicluster-data/ts-2/wal
uuid: "538dd97c7dc7490484e60c94a690ebe7"
format_stamp: "Formatted at 2026-08-06 03:47:08 on dist-test-slave-6ntk"
server_key: "2e79c48ad55622e3159e9f7352ceac7f"
server_key_iv: "d7960b16fa74add839378f7ab3fc5a4f"
server_key_version: "encryptionkey@0"
I20260806 03:47:08.093523 18574 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785988004637995-12279-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785988004637995-12279-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785988004637995-12279-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:47:08.110832 18574 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:47:08.112995 18574 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:47:08.113168 18574 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:47:08.113404 18574 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 03:47:08.113754 18574 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 03:47:08.113796 18574 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 03:47:08.113839 18574 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 03:47:08.113895 18574 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 03:47:08.119519 18574 rpc_server.cc:307] RPC server started. Bound to: 127.11.253.195:40133
I20260806 03:47:08.119590 18759 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.253.195:40133 every 8 connection(s)
I20260806 03:47:08.119870 18574 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785988004637995-12279-0/minicluster-data/ts-2/data/info.pb
I20260806 03:47:08.125190 12279 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskToVWRe/build/release/bin/kudu as pid 18574
I20260806 03:47:08.125332 12279 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785988004637995-12279-0/minicluster-data/ts-2/wal/instance
I20260806 03:47:08.125512 12279 external_mini_cluster.cc:1428] Setting key 0453eea0ff7c08c93fb4b55978e48655
Aug 06 03:47:08 dist-test-slave-6ntk krb5kdc[18008](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988028, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.253.195@KRBTEST.COM for kudu/127.11.253.254@KRBTEST.COM
I20260806 03:47:08.129452 18137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:08.120654 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.253.195:46789 (local address 127.11.253.254:46651)
0806 03:47:08.120697 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0806 03:47:08.120700 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 03:47:08.120893 (+ 193us) server_negotiation.cc:407] Connection header received
0806 03:47:08.121420 (+ 527us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:47:08.121421 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:47:08.121434 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:47:08.121491 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:47:08.122516 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:08.123019 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:08.124032 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:08.124230 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:08.124306 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:08.126736 (+ 2430us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:47:08.126737 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:47:08.126737 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:47:08.126744 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:47:08.128203 (+ 1459us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:47:08.128566 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:47:08.128567 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:47:08.128567 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:47:08.128614 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:47:08.128888 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:47:08.128889 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:47:08.128889 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:47:08.129003 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:47:08.129060 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0806 03:47:08.129289 (+ 229us) server_negotiation.cc:299] Negotiation successful
0806 03:47:08.129386 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260806 03:47:08.129622 18764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:08.120616 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:46651 (local address 127.11.253.195:46789)
0806 03:47:08.120771 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0806 03:47:08.120808 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0806 03:47:08.121315 (+ 507us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:47:08.121562 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:47:08.121566 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:47:08.121837 (+ 271us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:47:08.122416 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:08.122422 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:08.123117 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:08.123117 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:08.123939 (+ 822us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:08.123941 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:08.124315 (+ 374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:08.124316 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:08.124441 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:08.124949 (+ 508us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:47:08.124957 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:47:08.126654 (+ 1697us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:47:08.128280 (+ 1626us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:47:08.128287 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:47:08.128289 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:47:08.128506 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:47:08.128666 (+ 160us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:47:08.128666 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:47:08.128666 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:47:08.128826 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:47:08.129060 (+ 234us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:47:08.129064 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:47:08.129184 (+ 120us) client_negotiation.cc:769] Sending connection context
0806 03:47:08.129277 (+ 93us) client_negotiation.cc:240] Negotiation successful
0806 03:47:08.129412 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":70,"threads_started":1}
I20260806 03:47:08.130131 18760 heartbeater.cc:347] Connected to a master server at 127.11.253.254:46651
I20260806 03:47:08.130245 18760 heartbeater.cc:464] Registering TS with master...
I20260806 03:47:08.130410 18760 heartbeater.cc:511] Master 127.11.253.254:46651 requested a full tablet report, sending...
I20260806 03:47:08.130820 18075 ts_manager.cc:194] Registered new tserver with Master: 538dd97c7dc7490484e60c94a690ebe7 (127.11.253.195:40133)
I20260806 03:47:08.131168 18075 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.253.195@KRBTEST.COM'} at 127.11.253.195:46789
I20260806 03:47:08.137458 12279 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 03:47:08.144826 18137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:08.138901 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54808 (local address 127.11.253.254:46651)
0806 03:47:08.138968 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0806 03:47:08.138971 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 03:47:08.139110 (+ 139us) server_negotiation.cc:407] Connection header received
0806 03:47:08.139158 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:47:08.139159 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:47:08.139169 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:47:08.139219 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:47:08.139813 (+ 594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:08.140290 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:08.141021 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:08.141203 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:08.141251 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:08.142473 (+ 1222us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:47:08.142474 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:47:08.142474 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:47:08.142480 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:47:08.143890 (+ 1410us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:47:08.144176 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:47:08.144177 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:47:08.144177 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:47:08.144247 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:47:08.144396 (+ 149us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:47:08.144396 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:47:08.144397 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:47:08.144530 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:47:08.144607 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0806 03:47:08.144681 (+ 74us) server_negotiation.cc:299] Negotiation successful
0806 03:47:08.144739 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260806 03:47:08.146432 18075 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54808:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 03:47:08.148067 18075 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 03:47:08.154785 18779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:08.150609 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52416 (local address 127.11.253.193:33249)
0806 03:47:08.151065 (+ 456us) server_negotiation.cc:206] Beginning negotiation
0806 03:47:08.151069 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 03:47:08.151078 (+ 9us) server_negotiation.cc:407] Connection header received
0806 03:47:08.151116 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:47:08.151118 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:47:08.151133 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:47:08.151390 (+ 257us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 03:47:08.151480 (+ 90us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:08.152592 (+ 1112us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:08.153827 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:08.154382 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:08.154470 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:08.154531 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0806 03:47:08.154665 (+ 134us) server_negotiation.cc:299] Negotiation successful
0806 03:47:08.154706 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":418,"thread_start_us":119,"threads_started":1}
I20260806 03:47:08.154780 18776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:08.150405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.193:33249 (local address 127.0.0.1:52416)
0806 03:47:08.150585 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0806 03:47:08.150628 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 03:47:08.150694 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:47:08.151237 (+ 543us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:47:08.151239 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:47:08.151241 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 03:47:08.151297 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:08.151299 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:08.152704 (+ 1405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:08.152705 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:08.153732 (+ 1027us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:08.153734 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:08.154467 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:08.154468 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:08.154551 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:08.154565 (+ 14us) client_negotiation.cc:769] Sending connection context
0806 03:47:08.154631 (+ 66us) client_negotiation.cc:240] Negotiation successful
0806 03:47:08.154677 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"mutex_wait_us":1,"thread_start_us":68,"threads_started":1}
I20260806 03:47:08.154940 18782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:08.150849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43800 (local address 127.11.253.194:42607)
0806 03:47:08.151025 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0806 03:47:08.151029 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 03:47:08.151388 (+ 359us) server_negotiation.cc:407] Connection header received
0806 03:47:08.151433 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:47:08.151435 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:47:08.151450 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:47:08.151531 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 03:47:08.151612 (+ 81us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:08.152656 (+ 1044us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:08.153765 (+ 1109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:08.154475 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:08.154536 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:08.154592 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0806 03:47:08.154819 (+ 227us) server_negotiation.cc:299] Negotiation successful
0806 03:47:08.154858 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":90,"threads_started":1}
I20260806 03:47:08.155072 18781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:08.150722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.194:42607 (local address 127.0.0.1:43800)
0806 03:47:08.151268 (+ 546us) negotiation.cc:107] Waiting for socket to connect
0806 03:47:08.151279 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 03:47:08.151328 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:47:08.151528 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:47:08.151529 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:47:08.151531 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 03:47:08.151565 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:08.151567 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:08.152744 (+ 1177us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:08.152745 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:08.153672 (+ 927us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:08.153675 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:08.154624 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:08.154625 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:08.154674 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:08.154677 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 03:47:08.154724 (+ 47us) client_negotiation.cc:240] Negotiation successful
0806 03:47:08.154757 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":528,"thread_start_us":79,"threads_started":1}
I20260806 03:47:08.155071 18778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:08.150520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45466 (local address 127.11.253.195:40133)
0806 03:47:08.150982 (+ 462us) server_negotiation.cc:206] Beginning negotiation
0806 03:47:08.150987 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 03:47:08.150998 (+ 11us) server_negotiation.cc:407] Connection header received
0806 03:47:08.151042 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:47:08.151044 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:47:08.151060 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:47:08.151149 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 03:47:08.151356 (+ 207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:08.152204 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:08.153630 (+ 1426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:08.154397 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:08.154840 (+ 443us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:08.154913 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0806 03:47:08.154939 (+ 26us) server_negotiation.cc:299] Negotiation successful
0806 03:47:08.154984 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":425,"thread_start_us":94,"threads_started":1}
I20260806 03:47:08.154783 18777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:08.150406 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.195:40133 (local address 127.0.0.1:45466)
0806 03:47:08.150802 (+ 396us) negotiation.cc:107] Waiting for socket to connect
0806 03:47:08.150812 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 03:47:08.150863 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:47:08.151180 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:47:08.151183 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:47:08.151186 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 03:47:08.151262 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:08.151264 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:08.152307 (+ 1043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:08.152308 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:08.153544 (+ 1236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:08.153547 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:08.154505 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:08.154506 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:08.154566 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:08.154570 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 03:47:08.154616 (+ 46us) client_negotiation.cc:240] Negotiation successful
0806 03:47:08.154675 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":379,"mutex_wait_us":3,"thread_start_us":100,"threads_started":1}
I20260806 03:47:08.155732 18269 tablet_service.cc:1467] Processing CreateTablet for tablet e8dbbf719e294f868567ee45af6863ba (DEFAULT_TABLE table=test-table [id=a3730769bde5414cbdc536329fd5acea]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 03:47:08.155797 18476 tablet_service.cc:1467] Processing CreateTablet for tablet e8dbbf719e294f868567ee45af6863ba (DEFAULT_TABLE table=test-table [id=a3730769bde5414cbdc536329fd5acea]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 03:47:08.156157 18476 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e8dbbf719e294f868567ee45af6863ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:47:08.156175 18269 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e8dbbf719e294f868567ee45af6863ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:47:08.156190 18662 tablet_service.cc:1467] Processing CreateTablet for tablet e8dbbf719e294f868567ee45af6863ba (DEFAULT_TABLE table=test-table [id=a3730769bde5414cbdc536329fd5acea]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 03:47:08.156495 18662 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e8dbbf719e294f868567ee45af6863ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:47:08.158876 18783 tablet_bootstrap.cc:492] T e8dbbf719e294f868567ee45af6863ba P c27d8d93b40549e29a6bac8bd57d24ce: Bootstrap starting.
I20260806 03:47:08.159565 18783 tablet_bootstrap.cc:654] T e8dbbf719e294f868567ee45af6863ba P c27d8d93b40549e29a6bac8bd57d24ce: Neither blocks nor log segments found. Creating new log.
I20260806 03:47:08.159601 18788 tablet_bootstrap.cc:492] T e8dbbf719e294f868567ee45af6863ba P 2a1dc086aaeb45bb9876141dc3a08d8e: Bootstrap starting.
I20260806 03:47:08.159979 18789 tablet_bootstrap.cc:492] T e8dbbf719e294f868567ee45af6863ba P 538dd97c7dc7490484e60c94a690ebe7: Bootstrap starting.
I20260806 03:47:08.160321 18788 tablet_bootstrap.cc:654] T e8dbbf719e294f868567ee45af6863ba P 2a1dc086aaeb45bb9876141dc3a08d8e: Neither blocks nor log segments found. Creating new log.
I20260806 03:47:08.160439 18783 log.cc:824] T e8dbbf719e294f868567ee45af6863ba P c27d8d93b40549e29a6bac8bd57d24ce: Log is configured to *not* fsync() on all Append() calls
I20260806 03:47:08.160689 18789 tablet_bootstrap.cc:654] T e8dbbf719e294f868567ee45af6863ba P 538dd97c7dc7490484e60c94a690ebe7: Neither blocks nor log segments found. Creating new log.
I20260806 03:47:08.160966 18788 log.cc:824] T e8dbbf719e294f868567ee45af6863ba P 2a1dc086aaeb45bb9876141dc3a08d8e: Log is configured to *not* fsync() on all Append() calls
I20260806 03:47:08.161374 18789 log.cc:824] T e8dbbf719e294f868567ee45af6863ba P 538dd97c7dc7490484e60c94a690ebe7: Log is configured to *not* fsync() on all Append() calls
I20260806 03:47:08.161454 18783 tablet_bootstrap.cc:492] T e8dbbf719e294f868567ee45af6863ba P c27d8d93b40549e29a6bac8bd57d24ce: No bootstrap required, opened a new log
I20260806 03:47:08.161513 18783 ts_tablet_manager.cc:1397] T e8dbbf719e294f868567ee45af6863ba P c27d8d93b40549e29a6bac8bd57d24ce: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 03:47:08.161814 18788 tablet_bootstrap.cc:492] T e8dbbf719e294f868567ee45af6863ba P 2a1dc086aaeb45bb9876141dc3a08d8e: No bootstrap required, opened a new log
I20260806 03:47:08.161870 18788 ts_tablet_manager.cc:1397] T e8dbbf719e294f868567ee45af6863ba P 2a1dc086aaeb45bb9876141dc3a08d8e: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 03:47:08.162158 18789 tablet_bootstrap.cc:492] T e8dbbf719e294f868567ee45af6863ba P 538dd97c7dc7490484e60c94a690ebe7: No bootstrap required, opened a new log
I20260806 03:47:08.162218 18789 ts_tablet_manager.cc:1397] T e8dbbf719e294f868567ee45af6863ba P 538dd97c7dc7490484e60c94a690ebe7: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 03:47:08.163591 18788 raft_consensus.cc:348] T e8dbbf719e294f868567ee45af6863ba P 2a1dc086aaeb45bb9876141dc3a08d8e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "538dd97c7dc7490484e60c94a690ebe7" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 40133 } } peers { permanent_uuid: "c27d8d93b40549e29a6bac8bd57d24ce" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 33249 } } peers { permanent_uuid: "2a1dc086aaeb45bb9876141dc3a08d8e" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 42607 } }
I20260806 03:47:08.163712 18788 raft_consensus.cc:374] T e8dbbf719e294f868567ee45af6863ba P 2a1dc086aaeb45bb9876141dc3a08d8e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:47:08.163604 18783 raft_consensus.cc:348] T e8dbbf719e294f868567ee45af6863ba P c27d8d93b40549e29a6bac8bd57d24ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "538dd97c7dc7490484e60c94a690ebe7" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 40133 } } peers { permanent_uuid: "c27d8d93b40549e29a6bac8bd57d24ce" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 33249 } } peers { permanent_uuid: "2a1dc086aaeb45bb9876141dc3a08d8e" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 42607 } }
I20260806 03:47:08.163733 18788 raft_consensus.cc:729] T e8dbbf719e294f868567ee45af6863ba P 2a1dc086aaeb45bb9876141dc3a08d8e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a1dc086aaeb45bb9876141dc3a08d8e, State: Initialized, Role: FOLLOWER
I20260806 03:47:08.163758 18783 raft_consensus.cc:374] T e8dbbf719e294f868567ee45af6863ba P c27d8d93b40549e29a6bac8bd57d24ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:47:08.163792 18783 raft_consensus.cc:729] T e8dbbf719e294f868567ee45af6863ba P c27d8d93b40549e29a6bac8bd57d24ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c27d8d93b40549e29a6bac8bd57d24ce, State: Initialized, Role: FOLLOWER
I20260806 03:47:08.163846 18788 consensus_queue.cc:260] T e8dbbf719e294f868567ee45af6863ba P 2a1dc086aaeb45bb9876141dc3a08d8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "538dd97c7dc7490484e60c94a690ebe7" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 40133 } } peers { permanent_uuid: "c27d8d93b40549e29a6bac8bd57d24ce" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 33249 } } peers { permanent_uuid: "2a1dc086aaeb45bb9876141dc3a08d8e" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 42607 } }
I20260806 03:47:08.163913 18783 consensus_queue.cc:260] T e8dbbf719e294f868567ee45af6863ba P c27d8d93b40549e29a6bac8bd57d24ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "538dd97c7dc7490484e60c94a690ebe7" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 40133 } } peers { permanent_uuid: "c27d8d93b40549e29a6bac8bd57d24ce" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 33249 } } peers { permanent_uuid: "2a1dc086aaeb45bb9876141dc3a08d8e" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 42607 } }
I20260806 03:47:08.164145 18565 heartbeater.cc:503] Master 127.11.253.254:46651 was elected leader, sending a full tablet report...
I20260806 03:47:08.164144 18788 ts_tablet_manager.cc:1428] T e8dbbf719e294f868567ee45af6863ba P 2a1dc086aaeb45bb9876141dc3a08d8e: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260806 03:47:08.164194 18783 ts_tablet_manager.cc:1428] T e8dbbf719e294f868567ee45af6863ba P c27d8d93b40549e29a6bac8bd57d24ce: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 03:47:08.164304 18366 heartbeater.cc:503] Master 127.11.253.254:46651 was elected leader, sending a full tablet report...
I20260806 03:47:08.164893 18789 raft_consensus.cc:348] T e8dbbf719e294f868567ee45af6863ba P 538dd97c7dc7490484e60c94a690ebe7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "538dd97c7dc7490484e60c94a690ebe7" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 40133 } } peers { permanent_uuid: "c27d8d93b40549e29a6bac8bd57d24ce" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 33249 } } peers { permanent_uuid: "2a1dc086aaeb45bb9876141dc3a08d8e" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 42607 } }
I20260806 03:47:08.165045 18789 raft_consensus.cc:374] T e8dbbf719e294f868567ee45af6863ba P 538dd97c7dc7490484e60c94a690ebe7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:47:08.165091 18789 raft_consensus.cc:729] T e8dbbf719e294f868567ee45af6863ba P 538dd97c7dc7490484e60c94a690ebe7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 538dd97c7dc7490484e60c94a690ebe7, State: Initialized, Role: FOLLOWER
I20260806 03:47:08.165205 18789 consensus_queue.cc:260] T e8dbbf719e294f868567ee45af6863ba P 538dd97c7dc7490484e60c94a690ebe7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "538dd97c7dc7490484e60c94a690ebe7" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 40133 } } peers { permanent_uuid: "c27d8d93b40549e29a6bac8bd57d24ce" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 33249 } } peers { permanent_uuid: "2a1dc086aaeb45bb9876141dc3a08d8e" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 42607 } }
I20260806 03:47:08.165489 18789 ts_tablet_manager.cc:1428] T e8dbbf719e294f868567ee45af6863ba P 538dd97c7dc7490484e60c94a690ebe7: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260806 03:47:08.165512 18760 heartbeater.cc:503] Master 127.11.253.254:46651 was elected leader, sending a full tablet report...
W20260806 03:47:08.209354 18567 tablet.cc:2369] T e8dbbf719e294f868567ee45af6863ba P 2a1dc086aaeb45bb9876141dc3a08d8e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 03:47:08.299037 18367 tablet.cc:2369] T e8dbbf719e294f868567ee45af6863ba P c27d8d93b40549e29a6bac8bd57d24ce: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 03:47:08.370450 18763 tablet.cc:2369] T e8dbbf719e294f868567ee45af6863ba P 538dd97c7dc7490484e60c94a690ebe7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 03:47:08.391829 18796 raft_consensus.cc:482] T e8dbbf719e294f868567ee45af6863ba P c27d8d93b40549e29a6bac8bd57d24ce [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 03:47:08.391985 18796 raft_consensus.cc:504] T e8dbbf719e294f868567ee45af6863ba P c27d8d93b40549e29a6bac8bd57d24ce [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "538dd97c7dc7490484e60c94a690ebe7" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 40133 } } peers { permanent_uuid: "c27d8d93b40549e29a6bac8bd57d24ce" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 33249 } } peers { permanent_uuid: "2a1dc086aaeb45bb9876141dc3a08d8e" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 42607 } }
I20260806 03:47:08.392303 18796 leader_election.cc:288] T e8dbbf719e294f868567ee45af6863ba P c27d8d93b40549e29a6bac8bd57d24ce [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 538dd97c7dc7490484e60c94a690ebe7 (127.11.253.195:40133), 2a1dc086aaeb45bb9876141dc3a08d8e (127.11.253.194:42607)
I20260806 03:47:08.395449 18798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:08.392568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.195:40133 (local address 127.11.253.193:53259)
0806 03:47:08.392735 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0806 03:47:08.392749 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 03:47:08.392830 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:47:08.393014 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:47:08.393016 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:47:08.393019 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 03:47:08.393095 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:08.393096 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:08.393740 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:08.393741 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:08.394570 (+ 829us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:08.394572 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:08.395152 (+ 580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:08.395153 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:08.395227 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:08.395230 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 03:47:08.395292 (+ 62us) client_negotiation.cc:240] Negotiation successful
0806 03:47:08.395358 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":97,"threads_started":1}
I20260806 03:47:08.395445 18778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:08.392664 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.253.193:53259 (local address 127.11.253.195:40133)
0806 03:47:08.392737 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0806 03:47:08.392740 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 03:47:08.392829 (+ 89us) server_negotiation.cc:407] Connection header received
0806 03:47:08.392904 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:47:08.392905 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:47:08.392916 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:47:08.392997 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 03:47:08.393172 (+ 175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:08.393651 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:08.394664 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:08.395046 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:08.395151 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:08.395161 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0806 03:47:08.395296 (+ 135us) server_negotiation.cc:299] Negotiation successful
0806 03:47:08.395355 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260806 03:47:08.395849 18691 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8dbbf719e294f868567ee45af6863ba" candidate_uuid: "c27d8d93b40549e29a6bac8bd57d24ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "538dd97c7dc7490484e60c94a690ebe7" is_pre_election: true
I20260806 03:47:08.396039 18691 raft_consensus.cc:2454] T e8dbbf719e294f868567ee45af6863ba P 538dd97c7dc7490484e60c94a690ebe7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c27d8d93b40549e29a6bac8bd57d24ce in term 0.
I20260806 03:47:08.396044 18799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:08.392810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.194:42607 (local address 127.11.253.193:57265)
0806 03:47:08.392944 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0806 03:47:08.392956 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 03:47:08.393008 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:47:08.393252 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:47:08.393253 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:47:08.393255 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 03:47:08.393335 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:08.393337 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:08.394093 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:08.394094 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:08.394949 (+ 855us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:08.394952 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:08.395734 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:08.395736 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:08.395809 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:08.395812 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 03:47:08.395886 (+ 74us) client_negotiation.cc:240] Negotiation successful
0806 03:47:08.395932 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":50,"threads_started":1}
I20260806 03:47:08.396095 18782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:08.392915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.253.193:57265 (local address 127.11.253.194:42607)
0806 03:47:08.392990 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0806 03:47:08.392992 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:47:08.393005 (+ 13us) server_negotiation.cc:407] Connection header received
0806 03:47:08.393089 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:47:08.393091 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:47:08.393103 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:47:08.393190 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 03:47:08.393426 (+ 236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:08.393993 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:08.395059 (+ 1066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:08.395557 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:08.395665 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:08.395688 (+ 23us) server_negotiation.cc:1035] Waiting for connection context
0806 03:47:08.395934 (+ 246us) server_negotiation.cc:299] Negotiation successful
0806 03:47:08.395991 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260806 03:47:08.396369 18196 leader_election.cc:302] T e8dbbf719e294f868567ee45af6863ba P c27d8d93b40549e29a6bac8bd57d24ce [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 538dd97c7dc7490484e60c94a690ebe7, c27d8d93b40549e29a6bac8bd57d24ce; no voters:
I20260806 03:47:08.396502 18505 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8dbbf719e294f868567ee45af6863ba" candidate_uuid: "c27d8d93b40549e29a6bac8bd57d24ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a1dc086aaeb45bb9876141dc3a08d8e" is_pre_election: true
I20260806 03:47:08.396555 18796 raft_consensus.cc:2781] T e8dbbf719e294f868567ee45af6863ba P c27d8d93b40549e29a6bac8bd57d24ce [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 03:47:08.396639 18796 raft_consensus.cc:482] T e8dbbf719e294f868567ee45af6863ba P c27d8d93b40549e29a6bac8bd57d24ce [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 03:47:08.396669 18796 raft_consensus.cc:3037] T e8dbbf719e294f868567ee45af6863ba P c27d8d93b40549e29a6bac8bd57d24ce [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:47:08.396670 18505 raft_consensus.cc:2454] T e8dbbf719e294f868567ee45af6863ba P 2a1dc086aaeb45bb9876141dc3a08d8e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c27d8d93b40549e29a6bac8bd57d24ce in term 0.
I20260806 03:47:08.397531 18796 raft_consensus.cc:504] T e8dbbf719e294f868567ee45af6863ba P c27d8d93b40549e29a6bac8bd57d24ce [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "538dd97c7dc7490484e60c94a690ebe7" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 40133 } } peers { permanent_uuid: "c27d8d93b40549e29a6bac8bd57d24ce" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 33249 } } peers { permanent_uuid: "2a1dc086aaeb45bb9876141dc3a08d8e" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 42607 } }
I20260806 03:47:08.397640 18796 leader_election.cc:288] T e8dbbf719e294f868567ee45af6863ba P c27d8d93b40549e29a6bac8bd57d24ce [CANDIDATE]: Term 1 election: Requested vote from peers 538dd97c7dc7490484e60c94a690ebe7 (127.11.253.195:40133), 2a1dc086aaeb45bb9876141dc3a08d8e (127.11.253.194:42607)
I20260806 03:47:08.397924 18691 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8dbbf719e294f868567ee45af6863ba" candidate_uuid: "c27d8d93b40549e29a6bac8bd57d24ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "538dd97c7dc7490484e60c94a690ebe7"
I20260806 03:47:08.397914 18505 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8dbbf719e294f868567ee45af6863ba" candidate_uuid: "c27d8d93b40549e29a6bac8bd57d24ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a1dc086aaeb45bb9876141dc3a08d8e"
I20260806 03:47:08.397993 18691 raft_consensus.cc:3037] T e8dbbf719e294f868567ee45af6863ba P 538dd97c7dc7490484e60c94a690ebe7 [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:47:08.397996 18505 raft_consensus.cc:3037] T e8dbbf719e294f868567ee45af6863ba P 2a1dc086aaeb45bb9876141dc3a08d8e [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:47:08.398680 18691 raft_consensus.cc:2454] T e8dbbf719e294f868567ee45af6863ba P 538dd97c7dc7490484e60c94a690ebe7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c27d8d93b40549e29a6bac8bd57d24ce in term 1.
I20260806 03:47:08.398823 18505 raft_consensus.cc:2454] T e8dbbf719e294f868567ee45af6863ba P 2a1dc086aaeb45bb9876141dc3a08d8e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c27d8d93b40549e29a6bac8bd57d24ce in term 1.
I20260806 03:47:08.398895 18196 leader_election.cc:302] T e8dbbf719e294f868567ee45af6863ba P c27d8d93b40549e29a6bac8bd57d24ce [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 538dd97c7dc7490484e60c94a690ebe7, c27d8d93b40549e29a6bac8bd57d24ce; no voters:
I20260806 03:47:08.399012 18796 raft_consensus.cc:2781] T e8dbbf719e294f868567ee45af6863ba P c27d8d93b40549e29a6bac8bd57d24ce [term 1 FOLLOWER]: Leader election won for term 1
I20260806 03:47:08.399201 18796 raft_consensus.cc:686] T e8dbbf719e294f868567ee45af6863ba P c27d8d93b40549e29a6bac8bd57d24ce [term 1 LEADER]: Becoming Leader. State: Replica: c27d8d93b40549e29a6bac8bd57d24ce, State: Running, Role: LEADER
I20260806 03:47:08.399307 18796 consensus_queue.cc:237] T e8dbbf719e294f868567ee45af6863ba P c27d8d93b40549e29a6bac8bd57d24ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "538dd97c7dc7490484e60c94a690ebe7" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 40133 } } peers { permanent_uuid: "c27d8d93b40549e29a6bac8bd57d24ce" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 33249 } } peers { permanent_uuid: "2a1dc086aaeb45bb9876141dc3a08d8e" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 42607 } }
I20260806 03:47:08.400194 18071 catalog_manager.cc:5495] T e8dbbf719e294f868567ee45af6863ba P c27d8d93b40549e29a6bac8bd57d24ce reported cstate change: term changed from 0 to 1, leader changed from <none> to c27d8d93b40549e29a6bac8bd57d24ce (127.11.253.193). New cstate: current_term: 1 leader_uuid: "c27d8d93b40549e29a6bac8bd57d24ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "538dd97c7dc7490484e60c94a690ebe7" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 40133 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c27d8d93b40549e29a6bac8bd57d24ce" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 33249 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2a1dc086aaeb45bb9876141dc3a08d8e" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 42607 } health_report { overall_health: UNKNOWN } } }
I20260806 03:47:08.438625 18779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:08.436050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52426 (local address 127.11.253.193:33249)
0806 03:47:08.436125 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0806 03:47:08.436126 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 03:47:08.436134 (+ 8us) server_negotiation.cc:407] Connection header received
0806 03:47:08.436189 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:47:08.436190 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:47:08.436208 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:47:08.436277 (+ 69us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 03:47:08.436440 (+ 163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:08.436924 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:08.437753 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:08.437974 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:08.438098 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:08.438281 (+ 183us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 03:47:08.438391 (+ 110us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 03:47:08.438485 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0806 03:47:08.438510 (+ 25us) server_negotiation.cc:299] Negotiation successful
0806 03:47:08.438531 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260806 03:47:08.440613 18691 raft_consensus.cc:1261] T e8dbbf719e294f868567ee45af6863ba P 538dd97c7dc7490484e60c94a690ebe7 [term 1 FOLLOWER]: Refusing update from remote peer c27d8d93b40549e29a6bac8bd57d24ce: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 03:47:08.440646 18505 raft_consensus.cc:1261] T e8dbbf719e294f868567ee45af6863ba P 2a1dc086aaeb45bb9876141dc3a08d8e [term 1 FOLLOWER]: Refusing update from remote peer c27d8d93b40549e29a6bac8bd57d24ce: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 03:47:08.440949 18796 consensus_queue.cc:1035] T e8dbbf719e294f868567ee45af6863ba P c27d8d93b40549e29a6bac8bd57d24ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "538dd97c7dc7490484e60c94a690ebe7" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 40133 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 03:47:08.441093 18800 consensus_queue.cc:1035] T e8dbbf719e294f868567ee45af6863ba P c27d8d93b40549e29a6bac8bd57d24ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "2a1dc086aaeb45bb9876141dc3a08d8e" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 42607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 03:47:08.443478 18804 mvcc.cc:205] Tried to move back new op lower bound from 7315406964489404416 to 7315406964324077568. Current Snapshot: MvccSnapshot[applied={T|T < 7315406964489404416}]
I20260806 03:47:08.444397 18807 mvcc.cc:205] Tried to move back new op lower bound from 7315406964489404416 to 7315406964324077568. Current Snapshot: MvccSnapshot[applied={T|T < 7315406964489404416 or (T in {7315406964489404416})}]
I20260806 03:47:08.445809 18071 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54808:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 03:47:08.445943 18071 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54808:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 03:47:08.446663 18071 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 70ab0d51fcb04d96ba325ddb1b919ffd: Sending DeleteTablet for 3 replicas of tablet e8dbbf719e294f868567ee45af6863ba
I20260806 03:47:08.446971 18662 tablet_service.cc:1514] Processing DeleteTablet for tablet e8dbbf719e294f868567ee45af6863ba with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 03:47:08 UTC) from {username='kudu', principal='kudu/127.11.253.254@KRBTEST.COM'} at 127.0.0.1:45466
I20260806 03:47:08.446976 18269 tablet_service.cc:1514] Processing DeleteTablet for tablet e8dbbf719e294f868567ee45af6863ba with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 03:47:08 UTC) from {username='kudu', principal='kudu/127.11.253.254@KRBTEST.COM'} at 127.0.0.1:52416
I20260806 03:47:08.447055 18476 tablet_service.cc:1514] Processing DeleteTablet for tablet e8dbbf719e294f868567ee45af6863ba with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 03:47:08 UTC) from {username='kudu', principal='kudu/127.11.253.254@KRBTEST.COM'} at 127.0.0.1:43800
I20260806 03:47:08.447204 18817 tablet_replica.cc:330] T e8dbbf719e294f868567ee45af6863ba P c27d8d93b40549e29a6bac8bd57d24ce: stopping tablet replica
I20260806 03:47:08.447213 18818 tablet_replica.cc:330] T e8dbbf719e294f868567ee45af6863ba P 538dd97c7dc7490484e60c94a690ebe7: stopping tablet replica
I20260806 03:47:08.447306 18817 raft_consensus.cc:2229] T e8dbbf719e294f868567ee45af6863ba P c27d8d93b40549e29a6bac8bd57d24ce [term 1 LEADER]: Raft consensus shutting down.
I20260806 03:47:08.447306 18818 raft_consensus.cc:2229] T e8dbbf719e294f868567ee45af6863ba P 538dd97c7dc7490484e60c94a690ebe7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 03:47:08.447402 18817 raft_consensus.cc:2258] T e8dbbf719e294f868567ee45af6863ba P c27d8d93b40549e29a6bac8bd57d24ce [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 03:47:08.447409 18818 raft_consensus.cc:2258] T e8dbbf719e294f868567ee45af6863ba P 538dd97c7dc7490484e60c94a690ebe7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 03:47:08.447438 12279 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskToVWRe/build/release/bin/kudu with pid 18172
I20260806 03:47:08.449965 18818 ts_tablet_manager.cc:1905] T e8dbbf719e294f868567ee45af6863ba P 538dd97c7dc7490484e60c94a690ebe7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 03:47:08.451737 18818 ts_tablet_manager.cc:1918] T e8dbbf719e294f868567ee45af6863ba P 538dd97c7dc7490484e60c94a690ebe7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 03:47:08.451812 18818 log.cc:1196] T e8dbbf719e294f868567ee45af6863ba P 538dd97c7dc7490484e60c94a690ebe7: Deleting WAL directory at /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785988004637995-12279-0/minicluster-data/ts-2/wal/wals/e8dbbf719e294f868567ee45af6863ba
I20260806 03:47:08.452143 18818 ts_tablet_manager.cc:1939] T e8dbbf719e294f868567ee45af6863ba P 538dd97c7dc7490484e60c94a690ebe7: Deleting consensus metadata
I20260806 03:47:08.452661 18055 catalog_manager.cc:4841] TS 538dd97c7dc7490484e60c94a690ebe7 (127.11.253.195:40133): tablet e8dbbf719e294f868567ee45af6863ba (table test-table [id=a3730769bde5414cbdc536329fd5acea]) successfully deleted
I20260806 03:47:08.463552 18820 tablet_replica.cc:330] T e8dbbf719e294f868567ee45af6863ba P 2a1dc086aaeb45bb9876141dc3a08d8e: stopping tablet replica
I20260806 03:47:08.463706 18820 raft_consensus.cc:2229] T e8dbbf719e294f868567ee45af6863ba P 2a1dc086aaeb45bb9876141dc3a08d8e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 03:47:08.463832 18820 raft_consensus.cc:2258] T e8dbbf719e294f868567ee45af6863ba P 2a1dc086aaeb45bb9876141dc3a08d8e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 03:47:08.464579 18820 ts_tablet_manager.cc:1905] T e8dbbf719e294f868567ee45af6863ba P 2a1dc086aaeb45bb9876141dc3a08d8e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 03:47:08.466112 18820 ts_tablet_manager.cc:1918] T e8dbbf719e294f868567ee45af6863ba P 2a1dc086aaeb45bb9876141dc3a08d8e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 03:47:08.466189 18820 log.cc:1196] T e8dbbf719e294f868567ee45af6863ba P 2a1dc086aaeb45bb9876141dc3a08d8e: Deleting WAL directory at /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785988004637995-12279-0/minicluster-data/ts-1/wal/wals/e8dbbf719e294f868567ee45af6863ba
I20260806 03:47:08.466442 18820 ts_tablet_manager.cc:1939] T e8dbbf719e294f868567ee45af6863ba P 2a1dc086aaeb45bb9876141dc3a08d8e: Deleting consensus metadata
I20260806 03:47:08.466897 18055 catalog_manager.cc:4841] TS 2a1dc086aaeb45bb9876141dc3a08d8e (127.11.253.194:42607): tablet e8dbbf719e294f868567ee45af6863ba (table test-table [id=a3730769bde5414cbdc536329fd5acea]) successfully deleted
W20260806 03:47:08.468976 18054 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.253.193:33249 (error 108)
I20260806 03:47:08.469548 18777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:08.469409 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.193:33249 (local address 127.0.0.1:52428)
0806 03:47:08.469467 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0806 03:47:08.469503 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.253.193:33249: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260806 03:47:08.469635 12279 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskToVWRe/build/release/bin/kudu with pid 18377
W20260806 03:47:08.469704 18054 catalog_manager.cc:4584] TS c27d8d93b40549e29a6bac8bd57d24ce (127.11.253.193:33249): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e8dbbf719e294f868567ee45af6863ba: Network error: Client connection negotiation failed: client connection to 127.11.253.193:33249: connect: Connection refused (error 111)
I20260806 03:47:08.479933 12279 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskToVWRe/build/release/bin/kudu with pid 18574
I20260806 03:47:08.490578 12279 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskToVWRe/build/release/bin/kudu with pid 18028
2026-08-06T03:47:08Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (2430 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-06T03:47:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T03:47:08Z Disabled control of system clock
I20260806 03:47:08.523357 12279 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskToVWRe/build/release/bin/kudu
/tmp/dist-test-taskToVWRe/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785988004637995-12279-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785988004637995-12279-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785988004637995-12279-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785988004637995-12279-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.253.254:34471
--webserver_interface=127.11.253.254
--webserver_port=0
--builtin_ntp_servers=127.11.253.212:37753
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.253.254:34471
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260806 03:47:08.607874 18828 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:47:08.608085 18828 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:47:08.608107 18828 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:47:08.609267 18828 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 03:47:08.609314 18828 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:47:08.609328 18828 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:47:08.609347 18828 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 03:47:08.609360 18828 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 03:47:08.610796 18828 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.253.212:37753
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785988004637995-12279-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785988004637995-12279-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.253.254:34471
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.253.254:34471
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785988004637995-12279-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.253.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18828
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785988004637995-12279-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:47:08.611061 18828 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:47:08.611387 18828 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:47:08.616559 18834 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:47:08.616554 18838 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:47:08.616712 18828 server_base.cc:1034] running on GCE node
W20260806 03:47:08.616817 18835 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:47:08.617106 18828 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:47:08.617426 18828 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:47:08.618611 18828 hybrid_clock.cc:648] HybridClock initialized: now 1785988028618601 us; error 35 us; skew 500 ppm
I20260806 03:47:08.619954 18828 webserver.cc:460] Webserver started at http://127.11.253.254:37215/ using document root <none> and password file <none>
I20260806 03:47:08.620209 18828 fs_manager.cc:359] Metadata directory not provided
I20260806 03:47:08.620256 18828 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:47:08.620407 18828 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:47:08.621503 18828 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785988004637995-12279-0/minicluster-data/master-0/data/instance:
uuid: "5b6c1e142e8846ca9b6c9381eeb45564"
format_stamp: "Formatted at 2026-08-06 03:47:08 on dist-test-slave-6ntk"
server_key: "6e63520c53d75d6bdf405e1cebdcf521"
server_key_iv: "7f6721dd35217428efc1ca9759b8b207"
server_key_version: "encryptionkey@0"
I20260806 03:47:08.621922 18828 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785988004637995-12279-0/minicluster-data/master-0/wal/instance:
uuid: "5b6c1e142e8846ca9b6c9381eeb45564"
format_stamp: "Formatted at 2026-08-06 03:47:08 on dist-test-slave-6ntk"
server_key: "6e63520c53d75d6bdf405e1cebdcf521"
server_key_iv: "7f6721dd35217428efc1ca9759b8b207"
server_key_version: "encryptionkey@0"
I20260806 03:47:08.623284 18828 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 03:47:08.624068 18845 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:47:08.624284 18828 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 03:47:08.624377 18828 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785988004637995-12279-0/minicluster-data/master-0/data,/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785988004637995-12279-0/minicluster-data/master-0/wal
uuid: "5b6c1e142e8846ca9b6c9381eeb45564"
format_stamp: "Formatted at 2026-08-06 03:47:08 on dist-test-slave-6ntk"
server_key: "6e63520c53d75d6bdf405e1cebdcf521"
server_key_iv: "7f6721dd35217428efc1ca9759b8b207"
server_key_version: "encryptionkey@0"
I20260806 03:47:08.624446 18828 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785988004637995-12279-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785988004637995-12279-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785988004637995-12279-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:47:08.628264 18828 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:47:08.628557 18828 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:47:08.628690 18828 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:47:08.632913 18828 rpc_server.cc:307] RPC server started. Bound to: 127.11.253.254:34471
I20260806 03:47:08.632980 18921 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.253.254:34471 every 8 connection(s)
I20260806 03:47:08.633306 18828 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785988004637995-12279-0/minicluster-data/master-0/data/info.pb
I20260806 03:47:08.634089 18922 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:47:08.636952 18922 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5b6c1e142e8846ca9b6c9381eeb45564: Bootstrap starting.
I20260806 03:47:08.637542 18922 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5b6c1e142e8846ca9b6c9381eeb45564: Neither blocks nor log segments found. Creating new log.
I20260806 03:47:08.637861 18922 log.cc:824] T 00000000000000000000000000000000 P 5b6c1e142e8846ca9b6c9381eeb45564: Log is configured to *not* fsync() on all Append() calls
I20260806 03:47:08.637952 12279 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskToVWRe/build/release/bin/kudu as pid 18828
I20260806 03:47:08.638078 12279 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785988004637995-12279-0/minicluster-data/master-0/wal/instance
I20260806 03:47:08.638280 12279 external_mini_cluster.cc:1428] Setting key 4449782679fd7741f56a7436c1f6df0b
I20260806 03:47:08.638827 18922 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5b6c1e142e8846ca9b6c9381eeb45564: No bootstrap required, opened a new log
I20260806 03:47:08.640489 18922 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5b6c1e142e8846ca9b6c9381eeb45564 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b6c1e142e8846ca9b6c9381eeb45564" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 34471 } }
I20260806 03:47:08.640642 18922 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5b6c1e142e8846ca9b6c9381eeb45564 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:47:08.640688 18922 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5b6c1e142e8846ca9b6c9381eeb45564 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5b6c1e142e8846ca9b6c9381eeb45564, State: Initialized, Role: FOLLOWER
I20260806 03:47:08.640815 18922 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5b6c1e142e8846ca9b6c9381eeb45564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b6c1e142e8846ca9b6c9381eeb45564" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 34471 } }
I20260806 03:47:08.640913 18922 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5b6c1e142e8846ca9b6c9381eeb45564 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 03:47:08.640951 18922 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5b6c1e142e8846ca9b6c9381eeb45564 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 03:47:08.640992 18922 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5b6c1e142e8846ca9b6c9381eeb45564 [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:47:08.641577 18922 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5b6c1e142e8846ca9b6c9381eeb45564 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b6c1e142e8846ca9b6c9381eeb45564" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 34471 } }
I20260806 03:47:08.641705 18922 leader_election.cc:302] T 00000000000000000000000000000000 P 5b6c1e142e8846ca9b6c9381eeb45564 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5b6c1e142e8846ca9b6c9381eeb45564; no voters:
I20260806 03:47:08.641922 18922 leader_election.cc:288] T 00000000000000000000000000000000 P 5b6c1e142e8846ca9b6c9381eeb45564 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 03:47:08.641937 18928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:08.638803 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46200 (local address 127.11.253.254:34471)
0806 03:47:08.639037 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0806 03:47:08.639046 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 03:47:08.639075 (+ 29us) server_negotiation.cc:407] Connection header received
0806 03:47:08.640473 (+ 1398us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:47:08.640479 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:47:08.640518 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:47:08.640657 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:47:08.641385 (+ 728us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:47:08.641389 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:47:08.641389 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 03:47:08.641394 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:47:08.641442 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 03:47:08.641453 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:47:08.641523 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0806 03:47:08.641621 (+ 98us) server_negotiation.cc:299] Negotiation successful
0806 03:47:08.641683 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":74,"threads_started":1}
I20260806 03:47:08.642247 18922 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5b6c1e142e8846ca9b6c9381eeb45564 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 03:47:08.642287 18929 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5b6c1e142e8846ca9b6c9381eeb45564 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 03:47:08.642419 18929 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5b6c1e142e8846ca9b6c9381eeb45564 [term 1 LEADER]: Becoming Leader. State: Replica: 5b6c1e142e8846ca9b6c9381eeb45564, State: Running, Role: LEADER
I20260806 03:47:08.642506 18929 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5b6c1e142e8846ca9b6c9381eeb45564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b6c1e142e8846ca9b6c9381eeb45564" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 34471 } }
I20260806 03:47:08.642809 18929 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5b6c1e142e8846ca9b6c9381eeb45564 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5b6c1e142e8846ca9b6c9381eeb45564. Latest consensus state: current_term: 1 leader_uuid: "5b6c1e142e8846ca9b6c9381eeb45564" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b6c1e142e8846ca9b6c9381eeb45564" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 34471 } } }
I20260806 03:47:08.642900 18929 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5b6c1e142e8846ca9b6c9381eeb45564 [sys.catalog]: This master's current role is: LEADER
I20260806 03:47:08.642853 18931 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5b6c1e142e8846ca9b6c9381eeb45564 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5b6c1e142e8846ca9b6c9381eeb45564" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b6c1e142e8846ca9b6c9381eeb45564" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 34471 } } }
I20260806 03:47:08.642953 18931 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5b6c1e142e8846ca9b6c9381eeb45564 [sys.catalog]: This master's current role is: LEADER
I20260806 03:47:08.643164 18940 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 03:47:08.643693 18940 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 03:47:08.645222 18940 catalog_manager.cc:1347] Generated new cluster ID: cd9110e0bf584ccfb7de507f698b8c5e
I20260806 03:47:08.645277 18940 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 03:47:08.657437 18940 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 03:47:08.657601 18940 catalog_manager.cc:1504] Loading token signing keys...
I20260806 03:47:08.663591 18940 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5b6c1e142e8846ca9b6c9381eeb45564: Generated new TSK 0
I20260806 03:47:08.663796 18940 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 03:47:08.666953 12279 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskToVWRe/build/release/bin/kudu
/tmp/dist-test-taskToVWRe/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785988004637995-12279-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785988004637995-12279-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785988004637995-12279-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785988004637995-12279-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.253.193:0
--local_ip_for_outbound_sockets=127.11.253.193
--webserver_interface=127.11.253.193
--webserver_port=0
--tserver_master_addrs=127.11.253.254:34471
--builtin_ntp_servers=127.11.253.212:37753
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260806 03:47:08.752431 18954 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:47:08.752743 18954 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:47:08.752786 18954 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:47:08.754024 18954 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:47:08.754087 18954 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:47:08.754130 18954 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.253.193
I20260806 03:47:08.755527 18954 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.253.212:37753
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785988004637995-12279-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785988004637995-12279-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.253.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785988004637995-12279-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.253.193
--webserver_port=0
--tserver_master_addrs=127.11.253.254:34471
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18954
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.253.193
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785988004637995-12279-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:47:08.755796 18954 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:47:08.756108 18954 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:47:08.761396 18968 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:47:08.761281 18962 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:47:08.761278 18961 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:47:08.761618 18954 server_base.cc:1034] running on GCE node
I20260806 03:47:08.761816 18954 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:47:08.762120 18954 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:47:08.763275 18954 hybrid_clock.cc:648] HybridClock initialized: now 1785988028763264 us; error 29 us; skew 500 ppm
I20260806 03:47:08.764662 18954 webserver.cc:460] Webserver started at http://127.11.253.193:37755/ using document root <none> and password file <none>
I20260806 03:47:08.764928 18954 fs_manager.cc:359] Metadata directory not provided
I20260806 03:47:08.764974 18954 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:47:08.765125 18954 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:47:08.766343 18954 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785988004637995-12279-0/minicluster-data/ts-0/data/instance:
uuid: "9f4913b839174bc2980a6bf694093da5"
format_stamp: "Formatted at 2026-08-06 03:47:08 on dist-test-slave-6ntk"
server_key: "f145890a050e8fae6daa877ad03f7418"
server_key_iv: "629205a4506523328fd91ce0e8f9e7e3"
server_key_version: "encryptionkey@0"
I20260806 03:47:08.766705 18954 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785988004637995-12279-0/minicluster-data/ts-0/wal/instance:
uuid: "9f4913b839174bc2980a6bf694093da5"
format_stamp: "Formatted at 2026-08-06 03:47:08 on dist-test-slave-6ntk"
server_key: "f145890a050e8fae6daa877ad03f7418"
server_key_iv: "629205a4506523328fd91ce0e8f9e7e3"
server_key_version: "encryptionkey@0"
I20260806 03:47:08.768101 18954 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 03:47:08.768918 18977 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:47:08.769104 18954 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 03:47:08.769198 18954 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785988004637995-12279-0/minicluster-data/ts-0/data,/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785988004637995-12279-0/minicluster-data/ts-0/wal
uuid: "9f4913b839174bc2980a6bf694093da5"
format_stamp: "Formatted at 2026-08-06 03:47:08 on dist-test-slave-6ntk"
server_key: "f145890a050e8fae6daa877ad03f7418"
server_key_iv: "629205a4506523328fd91ce0e8f9e7e3"
server_key_version: "encryptionkey@0"
I20260806 03:47:08.769284 18954 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785988004637995-12279-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785988004637995-12279-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785988004637995-12279-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:47:08.772616 18954 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:47:08.772895 18954 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:47:08.773041 18954 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:47:08.773244 18954 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 03:47:08.773554 18954 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 03:47:08.773605 18954 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 03:47:08.773645 18954 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 03:47:08.773674 18954 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 03:47:08.779317 18954 rpc_server.cc:307] RPC server started. Bound to: 127.11.253.193:43253
I20260806 03:47:08.779392 19135 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.253.193:43253 every 8 connection(s)
I20260806 03:47:08.779711 18954 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785988004637995-12279-0/minicluster-data/ts-0/data/info.pb
I20260806 03:47:08.782830 18928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:08.780418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.253.193:40553 (local address 127.11.253.254:34471)
0806 03:47:08.780500 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0806 03:47:08.780502 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:47:08.780742 (+ 240us) server_negotiation.cc:407] Connection header received
0806 03:47:08.781207 (+ 465us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:47:08.781208 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:47:08.781215 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:47:08.781271 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:47:08.782623 (+ 1352us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:47:08.782625 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:47:08.782625 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 03:47:08.782629 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:47:08.782643 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 03:47:08.782646 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:47:08.782697 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0806 03:47:08.782756 (+ 59us) server_negotiation.cc:299] Negotiation successful
0806 03:47:08.782769 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260806 03:47:08.783037 12279 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskToVWRe/build/release/bin/kudu as pid 18954
I20260806 03:47:08.783072 19139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:08.780389 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:34471 (local address 127.11.253.193:40553)
0806 03:47:08.780620 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0806 03:47:08.780656 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0806 03:47:08.781125 (+ 469us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:47:08.781340 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:47:08.781344 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:47:08.781352 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:47:08.782485 (+ 1133us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 03:47:08.782485 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:47:08.782514 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 03:47:08.782542 (+ 28us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:47:08.782694 (+ 152us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:47:08.782696 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:47:08.782699 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 03:47:08.782756 (+ 57us) client_negotiation.cc:240] Negotiation successful
0806 03:47:08.782843 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":100,"threads_started":1}
I20260806 03:47:08.783134 12279 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785988004637995-12279-0/minicluster-data/ts-0/wal/instance
I20260806 03:47:08.783309 12279 external_mini_cluster.cc:1428] Setting key db6fa3202f24a5844780ad50fa155e32
I20260806 03:47:08.783607 19137 heartbeater.cc:347] Connected to a master server at 127.11.253.254:34471
I20260806 03:47:08.783708 19137 heartbeater.cc:464] Registering TS with master...
I20260806 03:47:08.783859 19137 heartbeater.cc:511] Master 127.11.253.254:34471 requested a full tablet report, sending...
I20260806 03:47:08.784297 12279 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskToVWRe/build/release/bin/kudu
/tmp/dist-test-taskToVWRe/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785988004637995-12279-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785988004637995-12279-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785988004637995-12279-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785988004637995-12279-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.253.194:0
--local_ip_for_outbound_sockets=127.11.253.194
--webserver_interface=127.11.253.194
--webserver_port=0
--tserver_master_addrs=127.11.253.254:34471
--builtin_ntp_servers=127.11.253.212:37753
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260806 03:47:08.784423 18873 ts_manager.cc:194] Registered new tserver with Master: 9f4913b839174bc2980a6bf694093da5 (127.11.253.193:43253)
W20260806 03:47:08.868645 19140 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:47:08.868880 19140 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:47:08.868903 19140 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:47:08.870182 19140 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:47:08.870226 19140 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:47:08.870254 19140 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.253.194
I20260806 03:47:08.871675 19140 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.253.212:37753
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785988004637995-12279-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785988004637995-12279-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.253.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785988004637995-12279-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.253.194
--webserver_port=0
--tserver_master_addrs=127.11.253.254:34471
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19140
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.253.194
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785988004637995-12279-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:47:08.871984 19140 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:47:08.872306 19140 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:47:08.877511 19152 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:47:08.877525 19149 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:47:08.877540 19150 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:47:08.877724 19140 server_base.cc:1034] running on GCE node
I20260806 03:47:08.878029 19140 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:47:08.878280 19140 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:47:08.879447 19140 hybrid_clock.cc:648] HybridClock initialized: now 1785988028879432 us; error 31 us; skew 500 ppm
I20260806 03:47:08.880834 19140 webserver.cc:460] Webserver started at http://127.11.253.194:37371/ using document root <none> and password file <none>
I20260806 03:47:08.881105 19140 fs_manager.cc:359] Metadata directory not provided
I20260806 03:47:08.881176 19140 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:47:08.881330 19140 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:47:08.882515 19140 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785988004637995-12279-0/minicluster-data/ts-1/data/instance:
uuid: "2c04083d12e344bbbda2c4f9b962ec0e"
format_stamp: "Formatted at 2026-08-06 03:47:08 on dist-test-slave-6ntk"
server_key: "d5f1ef8c455712056d4ec9d544a9f516"
server_key_iv: "72936edb258fc6e4b5a5f8d6692ab0b4"
server_key_version: "encryptionkey@0"
I20260806 03:47:08.882889 19140 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785988004637995-12279-0/minicluster-data/ts-1/wal/instance:
uuid: "2c04083d12e344bbbda2c4f9b962ec0e"
format_stamp: "Formatted at 2026-08-06 03:47:08 on dist-test-slave-6ntk"
server_key: "d5f1ef8c455712056d4ec9d544a9f516"
server_key_iv: "72936edb258fc6e4b5a5f8d6692ab0b4"
server_key_version: "encryptionkey@0"
I20260806 03:47:08.884328 19140 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260806 03:47:08.885144 19162 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:47:08.885330 19140 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 03:47:08.885434 19140 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785988004637995-12279-0/minicluster-data/ts-1/data,/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785988004637995-12279-0/minicluster-data/ts-1/wal
uuid: "2c04083d12e344bbbda2c4f9b962ec0e"
format_stamp: "Formatted at 2026-08-06 03:47:08 on dist-test-slave-6ntk"
server_key: "d5f1ef8c455712056d4ec9d544a9f516"
server_key_iv: "72936edb258fc6e4b5a5f8d6692ab0b4"
server_key_version: "encryptionkey@0"
I20260806 03:47:08.885509 19140 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785988004637995-12279-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785988004637995-12279-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785988004637995-12279-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:47:08.888998 19140 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:47:08.889263 19140 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:47:08.889410 19140 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:47:08.889607 19140 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 03:47:08.890004 19140 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 03:47:08.890053 19140 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 03:47:08.890094 19140 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 03:47:08.890123 19140 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 03:47:08.895838 19140 rpc_server.cc:307] RPC server started. Bound to: 127.11.253.194:36523
I20260806 03:47:08.895922 19322 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.253.194:36523 every 8 connection(s)
I20260806 03:47:08.896277 19140 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785988004637995-12279-0/minicluster-data/ts-1/data/info.pb
I20260806 03:47:08.899356 18928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:08.897081 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.253.194:59127 (local address 127.11.253.254:34471)
0806 03:47:08.897147 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0806 03:47:08.897149 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:47:08.897393 (+ 244us) server_negotiation.cc:407] Connection header received
0806 03:47:08.897833 (+ 440us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:47:08.897834 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:47:08.897840 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:47:08.897961 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:47:08.899170 (+ 1209us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:47:08.899171 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:47:08.899171 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 03:47:08.899174 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:47:08.899186 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 03:47:08.899189 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:47:08.899233 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0806 03:47:08.899290 (+ 57us) server_negotiation.cc:299] Negotiation successful
0806 03:47:08.899302 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260806 03:47:08.899561 19325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:08.897064 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:34471 (local address 127.11.253.194:59127)
0806 03:47:08.897263 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0806 03:47:08.897305 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 03:47:08.897751 (+ 446us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:47:08.898044 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:47:08.898047 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:47:08.898055 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:47:08.899040 (+ 985us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 03:47:08.899040 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:47:08.899068 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 03:47:08.899096 (+ 28us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:47:08.899232 (+ 136us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:47:08.899234 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:47:08.899236 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 03:47:08.899289 (+ 53us) client_negotiation.cc:240] Negotiation successful
0806 03:47:08.899336 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":87,"threads_started":1}
I20260806 03:47:08.900013 12279 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskToVWRe/build/release/bin/kudu as pid 19140
I20260806 03:47:08.900110 12279 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785988004637995-12279-0/minicluster-data/ts-1/wal/instance
I20260806 03:47:08.900113 19323 heartbeater.cc:347] Connected to a master server at 127.11.253.254:34471
I20260806 03:47:08.900221 19323 heartbeater.cc:464] Registering TS with master...
I20260806 03:47:08.900300 12279 external_mini_cluster.cc:1428] Setting key ffdbc5a66f7d382f4764e3ff6e83df3c
I20260806 03:47:08.900429 19323 heartbeater.cc:511] Master 127.11.253.254:34471 requested a full tablet report, sending...
I20260806 03:47:08.900786 18870 ts_manager.cc:194] Registered new tserver with Master: 2c04083d12e344bbbda2c4f9b962ec0e (127.11.253.194:36523)
I20260806 03:47:08.901286 12279 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskToVWRe/build/release/bin/kudu
/tmp/dist-test-taskToVWRe/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785988004637995-12279-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785988004637995-12279-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785988004637995-12279-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785988004637995-12279-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.253.195:0
--local_ip_for_outbound_sockets=127.11.253.195
--webserver_interface=127.11.253.195
--webserver_port=0
--tserver_master_addrs=127.11.253.254:34471
--builtin_ntp_servers=127.11.253.212:37753
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260806 03:47:08.984933 19326 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:47:08.985172 19326 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:47:08.985208 19326 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:47:08.986441 19326 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:47:08.986495 19326 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:47:08.986536 19326 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.253.195
I20260806 03:47:08.987847 19326 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.253.212:37753
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785988004637995-12279-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785988004637995-12279-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.253.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785988004637995-12279-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.253.195
--webserver_port=0
--tserver_master_addrs=127.11.253.254:34471
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19326
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.253.195
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785988004637995-12279-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:47:08.988159 19326 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:47:08.988471 19326 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:47:08.993485 19335 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:47:08.993497 19337 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:47:08.993500 19334 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:47:08.993736 19326 server_base.cc:1034] running on GCE node
I20260806 03:47:08.993992 19326 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:47:08.994185 19326 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:47:08.995318 19326 hybrid_clock.cc:648] HybridClock initialized: now 1785988028995291 us; error 41 us; skew 500 ppm
I20260806 03:47:08.996743 19326 webserver.cc:460] Webserver started at http://127.11.253.195:40231/ using document root <none> and password file <none>
I20260806 03:47:08.996999 19326 fs_manager.cc:359] Metadata directory not provided
I20260806 03:47:08.997045 19326 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:47:08.997192 19326 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:47:08.998481 19326 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785988004637995-12279-0/minicluster-data/ts-2/data/instance:
uuid: "630f4020dae646f7a02e8277481af92e"
format_stamp: "Formatted at 2026-08-06 03:47:08 on dist-test-slave-6ntk"
server_key: "78604eb701b14e6a7517c027049aaeee"
server_key_iv: "28cb3b2f9e61919f2473a2a88c988ce4"
server_key_version: "encryptionkey@0"
I20260806 03:47:08.998860 19326 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785988004637995-12279-0/minicluster-data/ts-2/wal/instance:
uuid: "630f4020dae646f7a02e8277481af92e"
format_stamp: "Formatted at 2026-08-06 03:47:08 on dist-test-slave-6ntk"
server_key: "78604eb701b14e6a7517c027049aaeee"
server_key_iv: "28cb3b2f9e61919f2473a2a88c988ce4"
server_key_version: "encryptionkey@0"
I20260806 03:47:09.000674 19326 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260806 03:47:09.001367 19347 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:47:09.001535 19326 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 03:47:09.001609 19326 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785988004637995-12279-0/minicluster-data/ts-2/data,/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785988004637995-12279-0/minicluster-data/ts-2/wal
uuid: "630f4020dae646f7a02e8277481af92e"
format_stamp: "Formatted at 2026-08-06 03:47:08 on dist-test-slave-6ntk"
server_key: "78604eb701b14e6a7517c027049aaeee"
server_key_iv: "28cb3b2f9e61919f2473a2a88c988ce4"
server_key_version: "encryptionkey@0"
I20260806 03:47:09.001662 19326 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785988004637995-12279-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785988004637995-12279-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785988004637995-12279-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:47:09.004966 19326 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:47:09.005194 19326 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:47:09.005288 19326 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:47:09.005438 19326 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 03:47:09.005681 19326 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 03:47:09.005710 19326 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 03:47:09.005730 19326 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 03:47:09.005743 19326 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 03:47:09.011605 19326 rpc_server.cc:307] RPC server started. Bound to: 127.11.253.195:39313
I20260806 03:47:09.011690 19516 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.253.195:39313 every 8 connection(s)
I20260806 03:47:09.012001 19326 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785988004637995-12279-0/minicluster-data/ts-2/data/info.pb
I20260806 03:47:09.015640 18928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:09.012908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.253.195:39693 (local address 127.11.253.254:34471)
0806 03:47:09.013095 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0806 03:47:09.013098 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 03:47:09.013416 (+ 318us) server_negotiation.cc:407] Connection header received
0806 03:47:09.014000 (+ 584us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:47:09.014001 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:47:09.014009 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:47:09.014075 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:47:09.015355 (+ 1280us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:47:09.015356 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:47:09.015357 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 03:47:09.015361 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:47:09.015377 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 03:47:09.015382 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:47:09.015457 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0806 03:47:09.015545 (+ 88us) server_negotiation.cc:299] Negotiation successful
0806 03:47:09.015563 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149}
I20260806 03:47:09.015864 19519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:09.012757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:34471 (local address 127.11.253.195:39693)
0806 03:47:09.013263 (+ 506us) negotiation.cc:107] Waiting for socket to connect
0806 03:47:09.013321 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0806 03:47:09.013820 (+ 499us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:47:09.014154 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:47:09.014158 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:47:09.014166 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:47:09.015167 (+ 1001us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 03:47:09.015168 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:47:09.015197 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 03:47:09.015226 (+ 29us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:47:09.015453 (+ 227us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:47:09.015460 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:47:09.015464 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 03:47:09.015557 (+ 93us) client_negotiation.cc:240] Negotiation successful
0806 03:47:09.015622 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":444,"thread_start_us":102,"threads_started":1}
I20260806 03:47:09.016443 19517 heartbeater.cc:347] Connected to a master server at 127.11.253.254:34471
I20260806 03:47:09.016578 19517 heartbeater.cc:464] Registering TS with master...
I20260806 03:47:09.016739 19517 heartbeater.cc:511] Master 127.11.253.254:34471 requested a full tablet report, sending...
I20260806 03:47:09.016837 12279 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskToVWRe/build/release/bin/kudu as pid 19326
I20260806 03:47:09.016949 12279 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785988004637995-12279-0/minicluster-data/ts-2/wal/instance
I20260806 03:47:09.017133 12279 external_mini_cluster.cc:1428] Setting key 524a649d2b9b64405f3dea0d2eb084c4
I20260806 03:47:09.017125 18870 ts_manager.cc:194] Registered new tserver with Master: 630f4020dae646f7a02e8277481af92e (127.11.253.195:39313)
I20260806 03:47:09.018079 12279 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 03:47:09.020447 18928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:09.019495 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46212 (local address 127.11.253.254:34471)
0806 03:47:09.019539 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0806 03:47:09.019541 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:47:09.019674 (+ 133us) server_negotiation.cc:407] Connection header received
0806 03:47:09.019723 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:47:09.019723 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:47:09.019727 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:47:09.019772 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:47:09.020234 (+ 462us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:47:09.020235 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:47:09.020235 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 03:47:09.020238 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:47:09.020250 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 03:47:09.020254 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:47:09.020295 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0806 03:47:09.020345 (+ 50us) server_negotiation.cc:299] Negotiation successful
0806 03:47:09.020355 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260806 03:47:09.022207 18870 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46212:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 03:47:09.023660 18870 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 03:47:09.027760 19529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:09.026002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.194:36523 (local address 127.0.0.1:54716)
0806 03:47:09.026316 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0806 03:47:09.026327 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 03:47:09.026378 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:47:09.026833 (+ 455us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:47:09.026839 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:47:09.026846 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:47:09.027333 (+ 487us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 03:47:09.027333 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:47:09.027349 (+ 16us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 03:47:09.027373 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:47:09.027590 (+ 217us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:47:09.027596 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:47:09.027599 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 03:47:09.027664 (+ 65us) client_negotiation.cc:240] Negotiation successful
0806 03:47:09.027678 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":300,"thread_start_us":95,"threads_started":1}
I20260806 03:47:09.028090 19534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:09.026416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54716 (local address 127.11.253.194:36523)
0806 03:47:09.026583 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0806 03:47:09.026587 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 03:47:09.026598 (+ 11us) server_negotiation.cc:407] Connection header received
0806 03:47:09.026695 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:47:09.026701 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:47:09.026713 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:47:09.027058 (+ 345us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:47:09.027450 (+ 392us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:47:09.027458 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:47:09.027458 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 03:47:09.027463 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:47:09.027499 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 03:47:09.027524 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:47:09.027589 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0806 03:47:09.027973 (+ 384us) server_negotiation.cc:299] Negotiation successful
0806 03:47:09.027992 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":102,"threads_started":1}
I20260806 03:47:09.028180 19533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:09.026317 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49284 (local address 127.11.253.195:39313)
0806 03:47:09.026611 (+ 294us) server_negotiation.cc:206] Beginning negotiation
0806 03:47:09.026615 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 03:47:09.026623 (+ 8us) server_negotiation.cc:407] Connection header received
0806 03:47:09.026645 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:47:09.026647 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:47:09.026654 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:47:09.026730 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:47:09.027771 (+ 1041us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:47:09.027778 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:47:09.027779 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 03:47:09.027784 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:47:09.027819 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 03:47:09.027844 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:47:09.027909 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0806 03:47:09.028065 (+ 156us) server_negotiation.cc:299] Negotiation successful
0806 03:47:09.028080 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":264,"thread_start_us":82,"threads_started":1}
I20260806 03:47:09.028257 19531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:09.026103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44210 (local address 127.11.253.193:43253)
0806 03:47:09.026514 (+ 411us) server_negotiation.cc:206] Beginning negotiation
0806 03:47:09.026519 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 03:47:09.026530 (+ 11us) server_negotiation.cc:407] Connection header received
0806 03:47:09.026558 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:47:09.026561 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:47:09.026571 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:47:09.026648 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:47:09.027783 (+ 1135us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:47:09.027790 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:47:09.027790 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 03:47:09.027794 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:47:09.027855 (+ 61us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 03:47:09.027887 (+ 32us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:47:09.027946 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0806 03:47:09.028139 (+ 193us) server_negotiation.cc:299] Negotiation successful
0806 03:47:09.028157 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":368,"thread_start_us":112,"threads_started":1}
I20260806 03:47:09.028087 19532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:09.026216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.195:39313 (local address 127.0.0.1:49284)
0806 03:47:09.026464 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0806 03:47:09.026472 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0806 03:47:09.026506 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:47:09.026986 (+ 480us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:47:09.026986 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:47:09.026989 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:47:09.027677 (+ 688us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 03:47:09.027678 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:47:09.027685 (+ 7us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 03:47:09.027697 (+ 12us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:47:09.027937 (+ 240us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:47:09.027938 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:47:09.027939 (+ 1us) client_negotiation.cc:769] Sending connection context
0806 03:47:09.027985 (+ 46us) client_negotiation.cc:240] Negotiation successful
0806 03:47:09.027997 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":233,"thread_start_us":69,"threads_started":1}
I20260806 03:47:09.028363 19530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:09.025975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.193:43253 (local address 127.0.0.1:44210)
0806 03:47:09.026154 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0806 03:47:09.026193 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0806 03:47:09.026252 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:47:09.026986 (+ 734us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:47:09.026986 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:47:09.026990 (+ 4us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:47:09.027584 (+ 594us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 03:47:09.027584 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:47:09.027590 (+ 6us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 03:47:09.027599 (+ 9us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:47:09.027973 (+ 374us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:47:09.027974 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:47:09.027974 (+ 0us) client_negotiation.cc:769] Sending connection context
0806 03:47:09.028017 (+ 43us) client_negotiation.cc:240] Negotiation successful
0806 03:47:09.028031 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":71,"threads_started":1}
I20260806 03:47:09.029080 19224 tablet_service.cc:1467] Processing CreateTablet for tablet 79464d2acc2b44a3bc5d65bdd6c7d555 (DEFAULT_TABLE table=test-table [id=318052c3dd08491ca9b27c8fea0e9031]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 03:47:09.029381 19224 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 79464d2acc2b44a3bc5d65bdd6c7d555. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:47:09.029392 19044 tablet_service.cc:1467] Processing CreateTablet for tablet 79464d2acc2b44a3bc5d65bdd6c7d555 (DEFAULT_TABLE table=test-table [id=318052c3dd08491ca9b27c8fea0e9031]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 03:47:09.029742 19044 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 79464d2acc2b44a3bc5d65bdd6c7d555. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:47:09.030295 19417 tablet_service.cc:1467] Processing CreateTablet for tablet 79464d2acc2b44a3bc5d65bdd6c7d555 (DEFAULT_TABLE table=test-table [id=318052c3dd08491ca9b27c8fea0e9031]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 03:47:09.030606 19417 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 79464d2acc2b44a3bc5d65bdd6c7d555. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:47:09.031989 19536 tablet_bootstrap.cc:492] T 79464d2acc2b44a3bc5d65bdd6c7d555 P 2c04083d12e344bbbda2c4f9b962ec0e: Bootstrap starting.
I20260806 03:47:09.032594 19536 tablet_bootstrap.cc:654] T 79464d2acc2b44a3bc5d65bdd6c7d555 P 2c04083d12e344bbbda2c4f9b962ec0e: Neither blocks nor log segments found. Creating new log.
I20260806 03:47:09.032891 19536 log.cc:824] T 79464d2acc2b44a3bc5d65bdd6c7d555 P 2c04083d12e344bbbda2c4f9b962ec0e: Log is configured to *not* fsync() on all Append() calls
I20260806 03:47:09.033092 19538 tablet_bootstrap.cc:492] T 79464d2acc2b44a3bc5d65bdd6c7d555 P 9f4913b839174bc2980a6bf694093da5: Bootstrap starting.
I20260806 03:47:09.033635 19538 tablet_bootstrap.cc:654] T 79464d2acc2b44a3bc5d65bdd6c7d555 P 9f4913b839174bc2980a6bf694093da5: Neither blocks nor log segments found. Creating new log.
I20260806 03:47:09.033754 19541 tablet_bootstrap.cc:492] T 79464d2acc2b44a3bc5d65bdd6c7d555 P 630f4020dae646f7a02e8277481af92e: Bootstrap starting.
I20260806 03:47:09.034013 19538 log.cc:824] T 79464d2acc2b44a3bc5d65bdd6c7d555 P 9f4913b839174bc2980a6bf694093da5: Log is configured to *not* fsync() on all Append() calls
I20260806 03:47:09.034444 19541 tablet_bootstrap.cc:654] T 79464d2acc2b44a3bc5d65bdd6c7d555 P 630f4020dae646f7a02e8277481af92e: Neither blocks nor log segments found. Creating new log.
I20260806 03:47:09.034698 19541 log.cc:824] T 79464d2acc2b44a3bc5d65bdd6c7d555 P 630f4020dae646f7a02e8277481af92e: Log is configured to *not* fsync() on all Append() calls
I20260806 03:47:09.034780 19538 tablet_bootstrap.cc:492] T 79464d2acc2b44a3bc5d65bdd6c7d555 P 9f4913b839174bc2980a6bf694093da5: No bootstrap required, opened a new log
I20260806 03:47:09.034780 19536 tablet_bootstrap.cc:492] T 79464d2acc2b44a3bc5d65bdd6c7d555 P 2c04083d12e344bbbda2c4f9b962ec0e: No bootstrap required, opened a new log
I20260806 03:47:09.034844 19536 ts_tablet_manager.cc:1397] T 79464d2acc2b44a3bc5d65bdd6c7d555 P 2c04083d12e344bbbda2c4f9b962ec0e: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260806 03:47:09.034844 19538 ts_tablet_manager.cc:1397] T 79464d2acc2b44a3bc5d65bdd6c7d555 P 9f4913b839174bc2980a6bf694093da5: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 03:47:09.035282 19541 tablet_bootstrap.cc:492] T 79464d2acc2b44a3bc5d65bdd6c7d555 P 630f4020dae646f7a02e8277481af92e: No bootstrap required, opened a new log
I20260806 03:47:09.035326 19541 ts_tablet_manager.cc:1397] T 79464d2acc2b44a3bc5d65bdd6c7d555 P 630f4020dae646f7a02e8277481af92e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 03:47:09.037304 19541 raft_consensus.cc:348] T 79464d2acc2b44a3bc5d65bdd6c7d555 P 630f4020dae646f7a02e8277481af92e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f4913b839174bc2980a6bf694093da5" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 43253 } } peers { permanent_uuid: "2c04083d12e344bbbda2c4f9b962ec0e" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 36523 } } peers { permanent_uuid: "630f4020dae646f7a02e8277481af92e" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 39313 } }
I20260806 03:47:09.037421 19541 raft_consensus.cc:374] T 79464d2acc2b44a3bc5d65bdd6c7d555 P 630f4020dae646f7a02e8277481af92e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:47:09.037442 19541 raft_consensus.cc:729] T 79464d2acc2b44a3bc5d65bdd6c7d555 P 630f4020dae646f7a02e8277481af92e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 630f4020dae646f7a02e8277481af92e, State: Initialized, Role: FOLLOWER
I20260806 03:47:09.037555 19541 consensus_queue.cc:260] T 79464d2acc2b44a3bc5d65bdd6c7d555 P 630f4020dae646f7a02e8277481af92e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f4913b839174bc2980a6bf694093da5" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 43253 } } peers { permanent_uuid: "2c04083d12e344bbbda2c4f9b962ec0e" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 36523 } } peers { permanent_uuid: "630f4020dae646f7a02e8277481af92e" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 39313 } }
I20260806 03:47:09.037483 19536 raft_consensus.cc:348] T 79464d2acc2b44a3bc5d65bdd6c7d555 P 2c04083d12e344bbbda2c4f9b962ec0e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f4913b839174bc2980a6bf694093da5" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 43253 } } peers { permanent_uuid: "2c04083d12e344bbbda2c4f9b962ec0e" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 36523 } } peers { permanent_uuid: "630f4020dae646f7a02e8277481af92e" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 39313 } }
I20260806 03:47:09.037483 19538 raft_consensus.cc:348] T 79464d2acc2b44a3bc5d65bdd6c7d555 P 9f4913b839174bc2980a6bf694093da5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f4913b839174bc2980a6bf694093da5" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 43253 } } peers { permanent_uuid: "2c04083d12e344bbbda2c4f9b962ec0e" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 36523 } } peers { permanent_uuid: "630f4020dae646f7a02e8277481af92e" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 39313 } }
I20260806 03:47:09.037637 19538 raft_consensus.cc:374] T 79464d2acc2b44a3bc5d65bdd6c7d555 P 9f4913b839174bc2980a6bf694093da5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:47:09.037635 19536 raft_consensus.cc:374] T 79464d2acc2b44a3bc5d65bdd6c7d555 P 2c04083d12e344bbbda2c4f9b962ec0e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:47:09.037671 19538 raft_consensus.cc:729] T 79464d2acc2b44a3bc5d65bdd6c7d555 P 9f4913b839174bc2980a6bf694093da5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f4913b839174bc2980a6bf694093da5, State: Initialized, Role: FOLLOWER
I20260806 03:47:09.037671 19536 raft_consensus.cc:729] T 79464d2acc2b44a3bc5d65bdd6c7d555 P 2c04083d12e344bbbda2c4f9b962ec0e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c04083d12e344bbbda2c4f9b962ec0e, State: Initialized, Role: FOLLOWER
I20260806 03:47:09.037789 19538 consensus_queue.cc:260] T 79464d2acc2b44a3bc5d65bdd6c7d555 P 9f4913b839174bc2980a6bf694093da5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f4913b839174bc2980a6bf694093da5" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 43253 } } peers { permanent_uuid: "2c04083d12e344bbbda2c4f9b962ec0e" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 36523 } } peers { permanent_uuid: "630f4020dae646f7a02e8277481af92e" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 39313 } }
I20260806 03:47:09.037797 19541 ts_tablet_manager.cc:1428] T 79464d2acc2b44a3bc5d65bdd6c7d555 P 630f4020dae646f7a02e8277481af92e: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260806 03:47:09.037849 19517 heartbeater.cc:503] Master 127.11.253.254:34471 was elected leader, sending a full tablet report...
I20260806 03:47:09.037827 19536 consensus_queue.cc:260] T 79464d2acc2b44a3bc5d65bdd6c7d555 P 2c04083d12e344bbbda2c4f9b962ec0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f4913b839174bc2980a6bf694093da5" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 43253 } } peers { permanent_uuid: "2c04083d12e344bbbda2c4f9b962ec0e" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 36523 } } peers { permanent_uuid: "630f4020dae646f7a02e8277481af92e" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 39313 } }
I20260806 03:47:09.038089 19538 ts_tablet_manager.cc:1428] T 79464d2acc2b44a3bc5d65bdd6c7d555 P 9f4913b839174bc2980a6bf694093da5: Time spent starting tablet: real 0.003s user 0.001s sys 0.002s
I20260806 03:47:09.038089 19536 ts_tablet_manager.cc:1428] T 79464d2acc2b44a3bc5d65bdd6c7d555 P 2c04083d12e344bbbda2c4f9b962ec0e: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260806 03:47:09.038127 19137 heartbeater.cc:503] Master 127.11.253.254:34471 was elected leader, sending a full tablet report...
I20260806 03:47:09.038203 19323 heartbeater.cc:503] Master 127.11.253.254:34471 was elected leader, sending a full tablet report...
W20260806 03:47:09.146826 19324 tablet.cc:2369] T 79464d2acc2b44a3bc5d65bdd6c7d555 P 2c04083d12e344bbbda2c4f9b962ec0e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 03:47:09.152359 19549 raft_consensus.cc:482] T 79464d2acc2b44a3bc5d65bdd6c7d555 P 9f4913b839174bc2980a6bf694093da5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 03:47:09.152493 19549 raft_consensus.cc:504] T 79464d2acc2b44a3bc5d65bdd6c7d555 P 9f4913b839174bc2980a6bf694093da5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f4913b839174bc2980a6bf694093da5" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 43253 } } peers { permanent_uuid: "2c04083d12e344bbbda2c4f9b962ec0e" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 36523 } } peers { permanent_uuid: "630f4020dae646f7a02e8277481af92e" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 39313 } }
I20260806 03:47:09.152901 19549 leader_election.cc:288] T 79464d2acc2b44a3bc5d65bdd6c7d555 P 9f4913b839174bc2980a6bf694093da5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2c04083d12e344bbbda2c4f9b962ec0e (127.11.253.194:36523), 630f4020dae646f7a02e8277481af92e (127.11.253.195:39313)
I20260806 03:47:09.154548 19533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:09.153410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.253.193:55835 (local address 127.11.253.195:39313)
0806 03:47:09.153468 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0806 03:47:09.153470 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:47:09.153484 (+ 14us) server_negotiation.cc:407] Connection header received
0806 03:47:09.153508 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:47:09.153508 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:47:09.153513 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:47:09.153561 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:47:09.154258 (+ 697us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:47:09.154260 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:47:09.154260 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 03:47:09.154265 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:47:09.154288 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 03:47:09.154293 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:47:09.154373 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0806 03:47:09.154459 (+ 86us) server_negotiation.cc:299] Negotiation successful
0806 03:47:09.154475 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260806 03:47:09.154548 19534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:09.153177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.253.193:34577 (local address 127.11.253.194:36523)
0806 03:47:09.153338 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0806 03:47:09.153340 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:47:09.153353 (+ 13us) server_negotiation.cc:407] Connection header received
0806 03:47:09.153414 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:47:09.153415 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:47:09.153422 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:47:09.153480 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:47:09.154271 (+ 791us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:47:09.154272 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:47:09.154273 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 03:47:09.154276 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:47:09.154293 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 03:47:09.154298 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:47:09.154395 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0806 03:47:09.154459 (+ 64us) server_negotiation.cc:299] Negotiation successful
0806 03:47:09.154475 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134}
I20260806 03:47:09.154556 19139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:09.153096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.194:36523 (local address 127.11.253.193:34577)
0806 03:47:09.153151 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0806 03:47:09.153164 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 03:47:09.153209 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:47:09.153540 (+ 331us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:47:09.153541 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:47:09.153543 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:47:09.154172 (+ 629us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 03:47:09.154173 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:47:09.154178 (+ 5us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 03:47:09.154187 (+ 9us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:47:09.154393 (+ 206us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:47:09.154393 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:47:09.154394 (+ 1us) client_negotiation.cc:769] Sending connection context
0806 03:47:09.154455 (+ 61us) client_negotiation.cc:240] Negotiation successful
0806 03:47:09.154471 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260806 03:47:09.154584 19553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:09.153234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.195:39313 (local address 127.11.253.193:55835)
0806 03:47:09.153398 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0806 03:47:09.153409 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 03:47:09.153462 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:47:09.153568 (+ 106us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:47:09.153569 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:47:09.153571 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:47:09.154104 (+ 533us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 03:47:09.154104 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:47:09.154112 (+ 8us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 03:47:09.154139 (+ 27us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:47:09.154369 (+ 230us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:47:09.154370 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:47:09.154371 (+ 1us) client_negotiation.cc:769] Sending connection context
0806 03:47:09.154457 (+ 86us) client_negotiation.cc:240] Negotiation successful
0806 03:47:09.154496 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"mutex_wait_us":25,"thread_start_us":103,"threads_started":1}
I20260806 03:47:09.154929 19443 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "79464d2acc2b44a3bc5d65bdd6c7d555" candidate_uuid: "9f4913b839174bc2980a6bf694093da5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "630f4020dae646f7a02e8277481af92e" is_pre_election: true
I20260806 03:47:09.154978 19258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "79464d2acc2b44a3bc5d65bdd6c7d555" candidate_uuid: "9f4913b839174bc2980a6bf694093da5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c04083d12e344bbbda2c4f9b962ec0e" is_pre_election: true
I20260806 03:47:09.155078 19443 raft_consensus.cc:2454] T 79464d2acc2b44a3bc5d65bdd6c7d555 P 630f4020dae646f7a02e8277481af92e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9f4913b839174bc2980a6bf694093da5 in term 0.
I20260806 03:47:09.155141 19258 raft_consensus.cc:2454] T 79464d2acc2b44a3bc5d65bdd6c7d555 P 2c04083d12e344bbbda2c4f9b962ec0e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9f4913b839174bc2980a6bf694093da5 in term 0.
I20260806 03:47:09.155352 18978 leader_election.cc:302] T 79464d2acc2b44a3bc5d65bdd6c7d555 P 9f4913b839174bc2980a6bf694093da5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 630f4020dae646f7a02e8277481af92e, 9f4913b839174bc2980a6bf694093da5; no voters:
I20260806 03:47:09.155503 19549 raft_consensus.cc:2781] T 79464d2acc2b44a3bc5d65bdd6c7d555 P 9f4913b839174bc2980a6bf694093da5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 03:47:09.155582 19549 raft_consensus.cc:482] T 79464d2acc2b44a3bc5d65bdd6c7d555 P 9f4913b839174bc2980a6bf694093da5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 03:47:09.155601 19549 raft_consensus.cc:3037] T 79464d2acc2b44a3bc5d65bdd6c7d555 P 9f4913b839174bc2980a6bf694093da5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:47:09.156366 19549 raft_consensus.cc:504] T 79464d2acc2b44a3bc5d65bdd6c7d555 P 9f4913b839174bc2980a6bf694093da5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f4913b839174bc2980a6bf694093da5" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 43253 } } peers { permanent_uuid: "2c04083d12e344bbbda2c4f9b962ec0e" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 36523 } } peers { permanent_uuid: "630f4020dae646f7a02e8277481af92e" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 39313 } }
I20260806 03:47:09.156492 19549 leader_election.cc:288] T 79464d2acc2b44a3bc5d65bdd6c7d555 P 9f4913b839174bc2980a6bf694093da5 [CANDIDATE]: Term 1 election: Requested vote from peers 2c04083d12e344bbbda2c4f9b962ec0e (127.11.253.194:36523), 630f4020dae646f7a02e8277481af92e (127.11.253.195:39313)
I20260806 03:47:09.156657 19258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "79464d2acc2b44a3bc5d65bdd6c7d555" candidate_uuid: "9f4913b839174bc2980a6bf694093da5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c04083d12e344bbbda2c4f9b962ec0e"
I20260806 03:47:09.156694 19443 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "79464d2acc2b44a3bc5d65bdd6c7d555" candidate_uuid: "9f4913b839174bc2980a6bf694093da5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "630f4020dae646f7a02e8277481af92e"
I20260806 03:47:09.156733 19258 raft_consensus.cc:3037] T 79464d2acc2b44a3bc5d65bdd6c7d555 P 2c04083d12e344bbbda2c4f9b962ec0e [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:47:09.156808 19443 raft_consensus.cc:3037] T 79464d2acc2b44a3bc5d65bdd6c7d555 P 630f4020dae646f7a02e8277481af92e [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:47:09.157464 19258 raft_consensus.cc:2454] T 79464d2acc2b44a3bc5d65bdd6c7d555 P 2c04083d12e344bbbda2c4f9b962ec0e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9f4913b839174bc2980a6bf694093da5 in term 1.
I20260806 03:47:09.157637 18978 leader_election.cc:302] T 79464d2acc2b44a3bc5d65bdd6c7d555 P 9f4913b839174bc2980a6bf694093da5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c04083d12e344bbbda2c4f9b962ec0e, 9f4913b839174bc2980a6bf694093da5; no voters:
I20260806 03:47:09.157729 19443 raft_consensus.cc:2454] T 79464d2acc2b44a3bc5d65bdd6c7d555 P 630f4020dae646f7a02e8277481af92e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9f4913b839174bc2980a6bf694093da5 in term 1.
I20260806 03:47:09.157747 19549 raft_consensus.cc:2781] T 79464d2acc2b44a3bc5d65bdd6c7d555 P 9f4913b839174bc2980a6bf694093da5 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 03:47:09.157922 19549 raft_consensus.cc:686] T 79464d2acc2b44a3bc5d65bdd6c7d555 P 9f4913b839174bc2980a6bf694093da5 [term 1 LEADER]: Becoming Leader. State: Replica: 9f4913b839174bc2980a6bf694093da5, State: Running, Role: LEADER
I20260806 03:47:09.158031 19549 consensus_queue.cc:237] T 79464d2acc2b44a3bc5d65bdd6c7d555 P 9f4913b839174bc2980a6bf694093da5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f4913b839174bc2980a6bf694093da5" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 43253 } } peers { permanent_uuid: "2c04083d12e344bbbda2c4f9b962ec0e" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 36523 } } peers { permanent_uuid: "630f4020dae646f7a02e8277481af92e" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 39313 } }
I20260806 03:47:09.158897 18873 catalog_manager.cc:5495] T 79464d2acc2b44a3bc5d65bdd6c7d555 P 9f4913b839174bc2980a6bf694093da5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9f4913b839174bc2980a6bf694093da5 (127.11.253.193). New cstate: current_term: 1 leader_uuid: "9f4913b839174bc2980a6bf694093da5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f4913b839174bc2980a6bf694093da5" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 43253 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2c04083d12e344bbbda2c4f9b962ec0e" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 36523 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "630f4020dae646f7a02e8277481af92e" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 39313 } health_report { overall_health: UNKNOWN } } }
I20260806 03:47:09.177971 19531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:09.176923 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44218 (local address 127.11.253.193:43253)
0806 03:47:09.176991 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0806 03:47:09.176994 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 03:47:09.177018 (+ 24us) server_negotiation.cc:407] Connection header received
0806 03:47:09.177058 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:47:09.177059 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:47:09.177074 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:47:09.177139 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:47:09.177692 (+ 553us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:47:09.177693 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:47:09.177694 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 03:47:09.177698 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:47:09.177713 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 03:47:09.177717 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:47:09.177782 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0806 03:47:09.177848 (+ 66us) server_negotiation.cc:299] Negotiation successful
0806 03:47:09.177864 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260806 03:47:09.179898 19443 raft_consensus.cc:1261] T 79464d2acc2b44a3bc5d65bdd6c7d555 P 630f4020dae646f7a02e8277481af92e [term 1 FOLLOWER]: Refusing update from remote peer 9f4913b839174bc2980a6bf694093da5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 03:47:09.179898 19258 raft_consensus.cc:1261] T 79464d2acc2b44a3bc5d65bdd6c7d555 P 2c04083d12e344bbbda2c4f9b962ec0e [term 1 FOLLOWER]: Refusing update from remote peer 9f4913b839174bc2980a6bf694093da5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 03:47:09.180272 19555 consensus_queue.cc:1035] T 79464d2acc2b44a3bc5d65bdd6c7d555 P 9f4913b839174bc2980a6bf694093da5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c04083d12e344bbbda2c4f9b962ec0e" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 36523 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 03:47:09.180409 19549 consensus_queue.cc:1035] T 79464d2acc2b44a3bc5d65bdd6c7d555 P 9f4913b839174bc2980a6bf694093da5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "630f4020dae646f7a02e8277481af92e" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 39313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 03:47:09.183763 19562 mvcc.cc:205] Tried to move back new op lower bound from 7315406967517548544 to 7315406967431950336. Current Snapshot: MvccSnapshot[applied={T|T < 7315406967517548544}]
I20260806 03:47:09.184881 18870 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:46212:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 03:47:09.184983 18870 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:46212:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 03:47:09.185704 18870 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5b6c1e142e8846ca9b6c9381eeb45564: Sending DeleteTablet for 3 replicas of tablet 79464d2acc2b44a3bc5d65bdd6c7d555
I20260806 03:47:09.186146 19224 tablet_service.cc:1514] Processing DeleteTablet for tablet 79464d2acc2b44a3bc5d65bdd6c7d555 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 03:47:09 UTC) from {username='slave'} at 127.0.0.1:54716
I20260806 03:47:09.186157 19417 tablet_service.cc:1514] Processing DeleteTablet for tablet 79464d2acc2b44a3bc5d65bdd6c7d555 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 03:47:09 UTC) from {username='slave'} at 127.0.0.1:49284
I20260806 03:47:09.186179 19044 tablet_service.cc:1514] Processing DeleteTablet for tablet 79464d2acc2b44a3bc5d65bdd6c7d555 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 03:47:09 UTC) from {username='slave'} at 127.0.0.1:44210
I20260806 03:47:09.186368 19571 tablet_replica.cc:330] T 79464d2acc2b44a3bc5d65bdd6c7d555 P 2c04083d12e344bbbda2c4f9b962ec0e: stopping tablet replica
I20260806 03:47:09.186455 12279 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskToVWRe/build/release/bin/kudu with pid 18954
I20260806 03:47:09.186555 19572 tablet_replica.cc:330] T 79464d2acc2b44a3bc5d65bdd6c7d555 P 9f4913b839174bc2980a6bf694093da5: stopping tablet replica
I20260806 03:47:09.186647 19572 raft_consensus.cc:2229] T 79464d2acc2b44a3bc5d65bdd6c7d555 P 9f4913b839174bc2980a6bf694093da5 [term 1 LEADER]: Raft consensus shutting down.
I20260806 03:47:09.186741 19572 raft_consensus.cc:2258] T 79464d2acc2b44a3bc5d65bdd6c7d555 P 9f4913b839174bc2980a6bf694093da5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 03:47:09.193107 19575 tablet_replica.cc:330] T 79464d2acc2b44a3bc5d65bdd6c7d555 P 630f4020dae646f7a02e8277481af92e: stopping tablet replica
I20260806 03:47:09.193234 19575 raft_consensus.cc:2229] T 79464d2acc2b44a3bc5d65bdd6c7d555 P 630f4020dae646f7a02e8277481af92e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 03:47:09.193341 19575 raft_consensus.cc:2258] T 79464d2acc2b44a3bc5d65bdd6c7d555 P 630f4020dae646f7a02e8277481af92e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 03:47:09.186488 19571 raft_consensus.cc:2229] T 79464d2acc2b44a3bc5d65bdd6c7d555 P 2c04083d12e344bbbda2c4f9b962ec0e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 03:47:09.194468 19571 raft_consensus.cc:2258] T 79464d2acc2b44a3bc5d65bdd6c7d555 P 2c04083d12e344bbbda2c4f9b962ec0e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 03:47:09.199648 19575 ts_tablet_manager.cc:1905] T 79464d2acc2b44a3bc5d65bdd6c7d555 P 630f4020dae646f7a02e8277481af92e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 03:47:09.200479 19571 ts_tablet_manager.cc:1905] T 79464d2acc2b44a3bc5d65bdd6c7d555 P 2c04083d12e344bbbda2c4f9b962ec0e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 03:47:09.201745 19571 ts_tablet_manager.cc:1918] T 79464d2acc2b44a3bc5d65bdd6c7d555 P 2c04083d12e344bbbda2c4f9b962ec0e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 03:47:09.201750 19575 ts_tablet_manager.cc:1918] T 79464d2acc2b44a3bc5d65bdd6c7d555 P 630f4020dae646f7a02e8277481af92e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 03:47:09.201828 19575 log.cc:1196] T 79464d2acc2b44a3bc5d65bdd6c7d555 P 630f4020dae646f7a02e8277481af92e: Deleting WAL directory at /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785988004637995-12279-0/minicluster-data/ts-2/wal/wals/79464d2acc2b44a3bc5d65bdd6c7d555
I20260806 03:47:09.201856 19571 log.cc:1196] T 79464d2acc2b44a3bc5d65bdd6c7d555 P 2c04083d12e344bbbda2c4f9b962ec0e: Deleting WAL directory at /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785988004637995-12279-0/minicluster-data/ts-1/wal/wals/79464d2acc2b44a3bc5d65bdd6c7d555
I20260806 03:47:09.202145 19575 ts_tablet_manager.cc:1939] T 79464d2acc2b44a3bc5d65bdd6c7d555 P 630f4020dae646f7a02e8277481af92e: Deleting consensus metadata
I20260806 03:47:09.202153 19571 ts_tablet_manager.cc:1939] T 79464d2acc2b44a3bc5d65bdd6c7d555 P 2c04083d12e344bbbda2c4f9b962ec0e: Deleting consensus metadata
I20260806 03:47:09.202626 18846 catalog_manager.cc:4841] TS 630f4020dae646f7a02e8277481af92e (127.11.253.195:39313): tablet 79464d2acc2b44a3bc5d65bdd6c7d555 (table test-table [id=318052c3dd08491ca9b27c8fea0e9031]) successfully deleted
I20260806 03:47:09.202754 18846 catalog_manager.cc:4841] TS 2c04083d12e344bbbda2c4f9b962ec0e (127.11.253.194:36523): tablet 79464d2acc2b44a3bc5d65bdd6c7d555 (table test-table [id=318052c3dd08491ca9b27c8fea0e9031]) successfully deleted
W20260806 03:47:09.205159 18847 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.253.193:43253 (error 108)
I20260806 03:47:09.205708 19532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:09.205583 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.193:43253 (local address 127.0.0.1:44226)
0806 03:47:09.205638 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0806 03:47:09.205678 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.253.193:43253: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260806 03:47:09.205808 12279 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskToVWRe/build/release/bin/kudu with pid 19140
W20260806 03:47:09.205909 18847 catalog_manager.cc:4584] TS 9f4913b839174bc2980a6bf694093da5 (127.11.253.193:43253): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 79464d2acc2b44a3bc5d65bdd6c7d555: Network error: Client connection negotiation failed: client connection to 127.11.253.193:43253: connect: Connection refused (error 111)
I20260806 03:47:09.216284 12279 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskToVWRe/build/release/bin/kudu with pid 19326
I20260806 03:47:09.228627 12279 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskToVWRe/build/release/bin/kudu with pid 18828
2026-08-06T03:47:09Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (739 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260806 03:47:09.254391 12279 mini_oidc.cc:152] Starting JWKS server
I20260806 03:47:09.255607 12279 webserver.cc:460] Webserver started at https://127.0.0.1:38597/ using document root <none> and password file <none>
I20260806 03:47:09.255848 12279 mini_oidc.cc:189] Starting OIDC Discovery server
I20260806 03:47:09.256078 12279 webserver.cc:460] Webserver started at http://127.0.0.1:37811/ using document root <none> and password file <none>
2026-08-06T03:47:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T03:47:09Z Disabled control of system clock
I20260806 03:47:09.265049 12279 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskToVWRe/build/release/bin/kudu
/tmp/dist-test-taskToVWRe/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785988004637995-12279-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785988004637995-12279-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785988004637995-12279-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785988004637995-12279-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.253.254:42983
--webserver_interface=127.11.253.254
--webserver_port=0
--builtin_ntp_servers=127.11.253.212:33935
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.253.254:42983
--enable_jwt_token_auth=true
--jwks_url=https://localhost:38597/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785988004637995-12279-0/testchainca.cert with env {}
W20260806 03:47:09.351444 19583 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:47:09.351742 19583 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:47:09.351778 19583 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:47:09.353132 19583 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 03:47:09.353178 19583 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:47:09.353190 19583 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:47:09.353204 19583 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 03:47:09.353240 19583 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260806 03:47:09.353266 19583 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260806 03:47:09.353286 19583 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:38597/jwks.json
I20260806 03:47:09.355028 19583 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.253.212:33935
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785988004637995-12279-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785988004637995-12279-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.253.254:42983
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.253.254:42983
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:38597/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785988004637995-12279-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.253.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785988004637995-12279-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19583
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785988004637995-12279-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:47:09.355306 19583 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:47:09.355538 19583 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:47:09.360473 19590 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:47:09.360549 19589 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:47:09.360495 19596 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:47:09.360782 19583 server_base.cc:1034] running on GCE node
I20260806 03:47:09.361056 19583 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:47:09.361387 19583 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:47:09.362527 19583 hybrid_clock.cc:648] HybridClock initialized: now 1785988029362510 us; error 32 us; skew 500 ppm
I20260806 03:47:09.363857 19583 webserver.cc:460] Webserver started at http://127.11.253.254:44379/ using document root <none> and password file <none>
I20260806 03:47:09.364100 19583 fs_manager.cc:359] Metadata directory not provided
I20260806 03:47:09.364147 19583 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:47:09.364295 19583 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:47:09.365427 19583 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785988004637995-12279-0/minicluster-data/master-0/data/instance:
uuid: "755abf28f43a4154a93775084c3c615a"
format_stamp: "Formatted at 2026-08-06 03:47:09 on dist-test-slave-6ntk"
server_key: "97bcbbc85fc759012e44b4a1776f64b1"
server_key_iv: "79076b9fbfc1cd65c84e78f59cc4c773"
server_key_version: "encryptionkey@0"
I20260806 03:47:09.365803 19583 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785988004637995-12279-0/minicluster-data/master-0/wal/instance:
uuid: "755abf28f43a4154a93775084c3c615a"
format_stamp: "Formatted at 2026-08-06 03:47:09 on dist-test-slave-6ntk"
server_key: "97bcbbc85fc759012e44b4a1776f64b1"
server_key_iv: "79076b9fbfc1cd65c84e78f59cc4c773"
server_key_version: "encryptionkey@0"
I20260806 03:47:09.367146 19583 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 03:47:09.367872 19602 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:47:09.368049 19583 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 03:47:09.368156 19583 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785988004637995-12279-0/minicluster-data/master-0/data,/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785988004637995-12279-0/minicluster-data/master-0/wal
uuid: "755abf28f43a4154a93775084c3c615a"
format_stamp: "Formatted at 2026-08-06 03:47:09 on dist-test-slave-6ntk"
server_key: "97bcbbc85fc759012e44b4a1776f64b1"
server_key_iv: "79076b9fbfc1cd65c84e78f59cc4c773"
server_key_version: "encryptionkey@0"
I20260806 03:47:09.368232 19583 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785988004637995-12279-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785988004637995-12279-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785988004637995-12279-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:47:09.378091 19583 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:47:09.378381 19583 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:47:09.378530 19583 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:47:09.382493 19583 rpc_server.cc:307] RPC server started. Bound to: 127.11.253.254:42983
I20260806 03:47:09.382527 19675 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.253.254:42983 every 8 connection(s)
I20260806 03:47:09.382862 19583 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785988004637995-12279-0/minicluster-data/master-0/data/info.pb
I20260806 03:47:09.383898 19676 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:47:09.386276 19676 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 755abf28f43a4154a93775084c3c615a: Bootstrap starting.
I20260806 03:47:09.386873 19676 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 755abf28f43a4154a93775084c3c615a: Neither blocks nor log segments found. Creating new log.
I20260806 03:47:09.387179 19676 log.cc:824] T 00000000000000000000000000000000 P 755abf28f43a4154a93775084c3c615a: Log is configured to *not* fsync() on all Append() calls
I20260806 03:47:09.387848 19676 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 755abf28f43a4154a93775084c3c615a: No bootstrap required, opened a new log
I20260806 03:47:09.389406 19676 raft_consensus.cc:348] T 00000000000000000000000000000000 P 755abf28f43a4154a93775084c3c615a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "755abf28f43a4154a93775084c3c615a" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 42983 } }
I20260806 03:47:09.389546 19676 raft_consensus.cc:374] T 00000000000000000000000000000000 P 755abf28f43a4154a93775084c3c615a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:47:09.389599 19676 raft_consensus.cc:729] T 00000000000000000000000000000000 P 755abf28f43a4154a93775084c3c615a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 755abf28f43a4154a93775084c3c615a, State: Initialized, Role: FOLLOWER
I20260806 03:47:09.389763 19676 consensus_queue.cc:260] T 00000000000000000000000000000000 P 755abf28f43a4154a93775084c3c615a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "755abf28f43a4154a93775084c3c615a" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 42983 } }
I20260806 03:47:09.389894 19676 raft_consensus.cc:388] T 00000000000000000000000000000000 P 755abf28f43a4154a93775084c3c615a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 03:47:09.389940 19676 raft_consensus.cc:482] T 00000000000000000000000000000000 P 755abf28f43a4154a93775084c3c615a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 03:47:09.389997 19676 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 755abf28f43a4154a93775084c3c615a [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:47:09.390050 12279 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskToVWRe/build/release/bin/kudu as pid 19583
I20260806 03:47:09.390149 12279 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785988004637995-12279-0/minicluster-data/master-0/wal/instance
I20260806 03:47:09.390328 12279 external_mini_cluster.cc:1428] Setting key bd9691e275ed732b046e9e8b5d454e9b
I20260806 03:47:09.390676 19676 raft_consensus.cc:504] T 00000000000000000000000000000000 P 755abf28f43a4154a93775084c3c615a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "755abf28f43a4154a93775084c3c615a" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 42983 } }
I20260806 03:47:09.390830 19676 leader_election.cc:302] T 00000000000000000000000000000000 P 755abf28f43a4154a93775084c3c615a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 755abf28f43a4154a93775084c3c615a; no voters:
I20260806 03:47:09.390992 19676 leader_election.cc:288] T 00000000000000000000000000000000 P 755abf28f43a4154a93775084c3c615a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 03:47:09.391079 19683 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 755abf28f43a4154a93775084c3c615a [term 1 FOLLOWER]: Leader election won for term 1
I20260806 03:47:09.391237 19676 sys_catalog.cc:564] T 00000000000000000000000000000000 P 755abf28f43a4154a93775084c3c615a [sys.catalog]: configured and running, proceeding with master startup.
I20260806 03:47:09.391265 19683 raft_consensus.cc:686] T 00000000000000000000000000000000 P 755abf28f43a4154a93775084c3c615a [term 1 LEADER]: Becoming Leader. State: Replica: 755abf28f43a4154a93775084c3c615a, State: Running, Role: LEADER
I20260806 03:47:09.391444 19683 consensus_queue.cc:237] T 00000000000000000000000000000000 P 755abf28f43a4154a93775084c3c615a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "755abf28f43a4154a93775084c3c615a" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 42983 } }
I20260806 03:47:09.391958 19686 sys_catalog.cc:455] T 00000000000000000000000000000000 P 755abf28f43a4154a93775084c3c615a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 755abf28f43a4154a93775084c3c615a. Latest consensus state: current_term: 1 leader_uuid: "755abf28f43a4154a93775084c3c615a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "755abf28f43a4154a93775084c3c615a" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 42983 } } }
I20260806 03:47:09.392076 19686 sys_catalog.cc:458] T 00000000000000000000000000000000 P 755abf28f43a4154a93775084c3c615a [sys.catalog]: This master's current role is: LEADER
I20260806 03:47:09.392453 19685 sys_catalog.cc:455] T 00000000000000000000000000000000 P 755abf28f43a4154a93775084c3c615a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "755abf28f43a4154a93775084c3c615a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "755abf28f43a4154a93775084c3c615a" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 42983 } } }
I20260806 03:47:09.392529 19685 sys_catalog.cc:458] T 00000000000000000000000000000000 P 755abf28f43a4154a93775084c3c615a [sys.catalog]: This master's current role is: LEADER
I20260806 03:47:09.392766 19696 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 03:47:09.393353 19696 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 03:47:09.394939 19696 catalog_manager.cc:1347] Generated new cluster ID: 1dbf77a566cc43c49d2ce209b3c10061
I20260806 03:47:09.394996 19696 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 03:47:09.396694 19682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:09.390814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40486 (local address 127.11.253.254:42983)
0806 03:47:09.391044 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0806 03:47:09.391054 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 03:47:09.391092 (+ 38us) server_negotiation.cc:407] Connection header received
0806 03:47:09.391209 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:47:09.391215 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:47:09.391286 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:47:09.391750 (+ 464us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:47:09.391861 (+ 111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:09.393249 (+ 1388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:09.394239 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:09.394870 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:09.395811 (+ 941us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:09.395969 (+ 158us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:47:09.395972 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:47:09.395973 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 03:47:09.396175 (+ 202us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:47:09.396215 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 03:47:09.396229 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:47:09.396299 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0806 03:47:09.396402 (+ 103us) server_negotiation.cc:299] Negotiation successful
0806 03:47:09.396484 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":157,"thread_start_us":96,"threads_started":1}
I20260806 03:47:09.397615 12279 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 03:47:09.398756 19696 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 03:47:09.399210 19696 catalog_manager.cc:1504] Loading token signing keys...
I20260806 03:47:09.410110 19696 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 755abf28f43a4154a93775084c3c615a: Generated new TSK 0
I20260806 03:47:09.410342 19696 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260806 03:47:09.435681 19720 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260806 03:47:09.435863 19720 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260806 03:47:09.436076 19682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:09.420376 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40496 (local address 127.11.253.254:42983)
0806 03:47:09.420450 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0806 03:47:09.420452 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:47:09.420594 (+ 142us) server_negotiation.cc:407] Connection header received
0806 03:47:09.420713 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:47:09.420714 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:47:09.420725 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:47:09.420792 (+ 67us) server_negotiation.cc:226] Negotiated authn=JWT
0806 03:47:09.420975 (+ 183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:09.421463 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:09.422613 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:09.422806 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:09.422864 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:09.423104 (+ 240us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0806 03:47:09.435825 (+ 12721us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38597/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0806 03:47:09.436010 (+ 185us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:40496: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38597/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":40}
W20260806 03:47:09.436143 19682 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:40496: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38597/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260806 03:47:09.436609 19719 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 03:47:09.420312 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:42983 (local address 127.0.0.1:40496)
0806 03:47:09.420519 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0806 03:47:09.420531 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 03:47:09.420605 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:47:09.420814 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:47:09.420815 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:47:09.420817 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0806 03:47:09.420908 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:09.420910 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:09.421542 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:09.421543 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:09.422527 (+ 984us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:09.422531 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:09.422883 (+ 352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:09.422884 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:09.422962 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:09.422988 (+ 26us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0806 03:47:09.436046 (+ 13058us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38597/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0806 03:47:09.436185 (+ 139us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.11.253.254:42983: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38597/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":112,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.11.253.254:42983: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38597/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260806 03:47:09.437047 12279 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskToVWRe/build/release/bin/kudu with pid 19583
2026-08-06T03:47:09Z chronyd exiting
I20260806 03:47:09.483480 12279 test_util.cc:175] -----------------------------------------------
I20260806 03:47:09.483569 12279 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785988004637995-12279-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (230 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260806 03:47:09.484472 12279 mini_oidc.cc:152] Starting JWKS server
I20260806 03:47:09.485169 12279 webserver.cc:460] Webserver started at https://127.0.0.1:39857/ using document root <none> and password file <none>
I20260806 03:47:09.485235 12279 mini_oidc.cc:189] Starting OIDC Discovery server
I20260806 03:47:09.485383 12279 webserver.cc:460] Webserver started at http://127.0.0.1:38947/ using document root <none> and password file <none>
2026-08-06T03:47:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T03:47:09Z Disabled control of system clock
I20260806 03:47:09.494449 12279 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskToVWRe/build/release/bin/kudu
/tmp/dist-test-taskToVWRe/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785988004637995-12279-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785988004637995-12279-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785988004637995-12279-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785988004637995-12279-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.253.254:34437
--webserver_interface=127.11.253.254
--webserver_port=0
--builtin_ntp_servers=127.11.253.212:40183
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.253.254:34437
--enable_jwt_token_auth=true
--jwks_url=https://localhost:39857/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785988004637995-12279-0/testchainca.cert with env {}
W20260806 03:47:09.576692 19728 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:47:09.576884 19728 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:47:09.576907 19728 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:47:09.578148 19728 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 03:47:09.578192 19728 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:47:09.578207 19728 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:47:09.578219 19728 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 03:47:09.578231 19728 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260806 03:47:09.578243 19728 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260806 03:47:09.578286 19728 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:39857/jwks.json
I20260806 03:47:09.579751 19728 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.253.212:40183
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785988004637995-12279-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785988004637995-12279-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.253.254:34437
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.253.254:34437
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:39857/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785988004637995-12279-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.253.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785988004637995-12279-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19728
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785988004637995-12279-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:47:09.579999 19728 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:47:09.580205 19728 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:47:09.585412 19739 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:47:09.585434 19737 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:47:09.585553 19728 server_base.cc:1034] running on GCE node
W20260806 03:47:09.585417 19736 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:47:09.586002 19728 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:47:09.586303 19728 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:47:09.587468 19728 hybrid_clock.cc:648] HybridClock initialized: now 1785988029587452 us; error 34 us; skew 500 ppm
I20260806 03:47:09.588745 19728 webserver.cc:460] Webserver started at http://127.11.253.254:35643/ using document root <none> and password file <none>
I20260806 03:47:09.588984 19728 fs_manager.cc:359] Metadata directory not provided
I20260806 03:47:09.589030 19728 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:47:09.589164 19728 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:47:09.590278 19728 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785988004637995-12279-0/minicluster-data/master-0/data/instance:
uuid: "9804b81c635845d09c0ed50fc42fc4a1"
format_stamp: "Formatted at 2026-08-06 03:47:09 on dist-test-slave-6ntk"
server_key: "b9c5a5c1250966dd23e98d294e37a1d8"
server_key_iv: "e17763584f8bf2a1a7062b86587974f5"
server_key_version: "encryptionkey@0"
I20260806 03:47:09.590633 19728 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785988004637995-12279-0/minicluster-data/master-0/wal/instance:
uuid: "9804b81c635845d09c0ed50fc42fc4a1"
format_stamp: "Formatted at 2026-08-06 03:47:09 on dist-test-slave-6ntk"
server_key: "b9c5a5c1250966dd23e98d294e37a1d8"
server_key_iv: "e17763584f8bf2a1a7062b86587974f5"
server_key_version: "encryptionkey@0"
I20260806 03:47:09.591964 19728 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 03:47:09.592772 19749 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:47:09.592986 19728 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 03:47:09.593078 19728 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785988004637995-12279-0/minicluster-data/master-0/data,/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785988004637995-12279-0/minicluster-data/master-0/wal
uuid: "9804b81c635845d09c0ed50fc42fc4a1"
format_stamp: "Formatted at 2026-08-06 03:47:09 on dist-test-slave-6ntk"
server_key: "b9c5a5c1250966dd23e98d294e37a1d8"
server_key_iv: "e17763584f8bf2a1a7062b86587974f5"
server_key_version: "encryptionkey@0"
I20260806 03:47:09.593150 19728 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785988004637995-12279-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785988004637995-12279-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785988004637995-12279-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:47:09.618980 19728 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:47:09.619329 19728 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:47:09.619515 19728 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:47:09.624017 19728 rpc_server.cc:307] RPC server started. Bound to: 127.11.253.254:34437
I20260806 03:47:09.624051 19816 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.253.254:34437 every 8 connection(s)
I20260806 03:47:09.624433 19728 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785988004637995-12279-0/minicluster-data/master-0/data/info.pb
I20260806 03:47:09.625092 19818 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:47:09.627935 19818 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9804b81c635845d09c0ed50fc42fc4a1: Bootstrap starting.
I20260806 03:47:09.628490 19818 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9804b81c635845d09c0ed50fc42fc4a1: Neither blocks nor log segments found. Creating new log.
I20260806 03:47:09.628726 19818 log.cc:824] T 00000000000000000000000000000000 P 9804b81c635845d09c0ed50fc42fc4a1: Log is configured to *not* fsync() on all Append() calls
I20260806 03:47:09.629283 12279 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskToVWRe/build/release/bin/kudu as pid 19728
I20260806 03:47:09.629386 12279 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785988004637995-12279-0/minicluster-data/master-0/wal/instance
I20260806 03:47:09.629503 19818 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9804b81c635845d09c0ed50fc42fc4a1: No bootstrap required, opened a new log
I20260806 03:47:09.629577 12279 external_mini_cluster.cc:1428] Setting key 93ef8feb0f234cf709c3a703641d8bf2
I20260806 03:47:09.632095 19818 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9804b81c635845d09c0ed50fc42fc4a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9804b81c635845d09c0ed50fc42fc4a1" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 34437 } }
I20260806 03:47:09.632273 19818 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9804b81c635845d09c0ed50fc42fc4a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:47:09.632313 19818 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9804b81c635845d09c0ed50fc42fc4a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9804b81c635845d09c0ed50fc42fc4a1, State: Initialized, Role: FOLLOWER
I20260806 03:47:09.632452 19818 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9804b81c635845d09c0ed50fc42fc4a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9804b81c635845d09c0ed50fc42fc4a1" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 34437 } }
I20260806 03:47:09.632534 19818 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9804b81c635845d09c0ed50fc42fc4a1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 03:47:09.632566 19818 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9804b81c635845d09c0ed50fc42fc4a1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 03:47:09.632612 19818 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9804b81c635845d09c0ed50fc42fc4a1 [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:47:09.633291 19818 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9804b81c635845d09c0ed50fc42fc4a1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9804b81c635845d09c0ed50fc42fc4a1" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 34437 } }
I20260806 03:47:09.633411 19818 leader_election.cc:302] T 00000000000000000000000000000000 P 9804b81c635845d09c0ed50fc42fc4a1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9804b81c635845d09c0ed50fc42fc4a1; no voters:
I20260806 03:47:09.633600 19818 leader_election.cc:288] T 00000000000000000000000000000000 P 9804b81c635845d09c0ed50fc42fc4a1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 03:47:09.633625 19826 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9804b81c635845d09c0ed50fc42fc4a1 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 03:47:09.633739 19826 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9804b81c635845d09c0ed50fc42fc4a1 [term 1 LEADER]: Becoming Leader. State: Replica: 9804b81c635845d09c0ed50fc42fc4a1, State: Running, Role: LEADER
I20260806 03:47:09.633836 19826 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9804b81c635845d09c0ed50fc42fc4a1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9804b81c635845d09c0ed50fc42fc4a1" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 34437 } }
I20260806 03:47:09.633893 19818 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9804b81c635845d09c0ed50fc42fc4a1 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 03:47:09.634572 19826 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9804b81c635845d09c0ed50fc42fc4a1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9804b81c635845d09c0ed50fc42fc4a1. Latest consensus state: current_term: 1 leader_uuid: "9804b81c635845d09c0ed50fc42fc4a1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9804b81c635845d09c0ed50fc42fc4a1" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 34437 } } }
I20260806 03:47:09.634581 19827 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9804b81c635845d09c0ed50fc42fc4a1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9804b81c635845d09c0ed50fc42fc4a1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9804b81c635845d09c0ed50fc42fc4a1" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 34437 } } }
I20260806 03:47:09.634673 19826 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9804b81c635845d09c0ed50fc42fc4a1 [sys.catalog]: This master's current role is: LEADER
I20260806 03:47:09.634691 19827 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9804b81c635845d09c0ed50fc42fc4a1 [sys.catalog]: This master's current role is: LEADER
I20260806 03:47:09.634871 19838 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 03:47:09.635450 19838 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 03:47:09.635893 19824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:09.630123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47138 (local address 127.11.253.254:34437)
0806 03:47:09.630402 (+ 279us) server_negotiation.cc:206] Beginning negotiation
0806 03:47:09.630415 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 03:47:09.630441 (+ 26us) server_negotiation.cc:407] Connection header received
0806 03:47:09.630895 (+ 454us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:47:09.630902 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:47:09.630958 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:47:09.631071 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:47:09.631286 (+ 215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:09.632125 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:09.633220 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:09.633773 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:09.634222 (+ 449us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:09.634665 (+ 443us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:47:09.634669 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:47:09.634669 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 03:47:09.634884 (+ 215us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:47:09.634932 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 03:47:09.634947 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:47:09.635010 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0806 03:47:09.635346 (+ 336us) server_negotiation.cc:299] Negotiation successful
0806 03:47:09.635425 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":198,"thread_start_us":158,"threads_started":1}
I20260806 03:47:09.636831 12279 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 03:47:09.637286 19838 catalog_manager.cc:1347] Generated new cluster ID: 26b4737da19a4e18ae2468c417513366
I20260806 03:47:09.637354 19838 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 03:47:09.653568 19838 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 03:47:09.654178 19838 catalog_manager.cc:1504] Loading token signing keys...
I20260806 03:47:09.657265 19838 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9804b81c635845d09c0ed50fc42fc4a1: Generated new TSK 0
I20260806 03:47:09.657454 19838 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260806 03:47:09.680931 19859 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260806 03:47:09.681034 19859 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260806 03:47:09.681285 19824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:09.665635 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47150 (local address 127.11.253.254:34437)
0806 03:47:09.665679 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0806 03:47:09.665681 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:47:09.665783 (+ 102us) server_negotiation.cc:407] Connection header received
0806 03:47:09.665851 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:47:09.665852 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:47:09.665893 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:47:09.665970 (+ 77us) server_negotiation.cc:226] Negotiated authn=JWT
0806 03:47:09.666097 (+ 127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:09.666892 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:09.667710 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:09.668052 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:09.668116 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:09.668341 (+ 225us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0806 03:47:09.681072 (+ 12731us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39857/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0806 03:47:09.681214 (+ 142us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:47150: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39857/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":23}
W20260806 03:47:09.681349 19824 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:47150: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39857/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260806 03:47:09.681429 19857 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 03:47:09.665550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:34437 (local address 127.0.0.1:47150)
0806 03:47:09.665706 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0806 03:47:09.665717 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 03:47:09.665782 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:47:09.665976 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:47:09.665976 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:47:09.665978 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0806 03:47:09.666032 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:09.666034 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:09.666986 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:09.666987 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:09.667632 (+ 645us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:09.667634 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:09.668138 (+ 504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:09.668139 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:09.668194 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:09.668199 (+ 5us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0806 03:47:09.681195 (+ 12996us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39857/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0806 03:47:09.681297 (+ 102us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.11.253.254:34437: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39857/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":103,"threads_started":1}
I20260806 03:47:09.681813 12279 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskToVWRe/build/release/bin/kudu with pid 19728
2026-08-06T03:47:09Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (240 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260806 03:47:09.725136 12279 mini_oidc.cc:152] Starting JWKS server
I20260806 03:47:09.725848 12279 webserver.cc:460] Webserver started at https://127.0.0.1:37391/ using document root <none> and password file <none>
I20260806 03:47:09.725950 12279 mini_oidc.cc:189] Starting OIDC Discovery server
I20260806 03:47:09.726133 12279 webserver.cc:460] Webserver started at http://127.0.0.1:45825/ using document root <none> and password file <none>
2026-08-06T03:47:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T03:47:09Z Disabled control of system clock
I20260806 03:47:09.735401 12279 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskToVWRe/build/release/bin/kudu
/tmp/dist-test-taskToVWRe/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785988004637995-12279-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785988004637995-12279-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785988004637995-12279-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785988004637995-12279-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.253.254:34835
--webserver_interface=127.11.253.254
--webserver_port=0
--builtin_ntp_servers=127.11.253.212:39239
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.253.254:34835
--enable_jwt_token_auth=true
--jwks_url=https://localhost:37391/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260806 03:47:09.819893 19870 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:47:09.820112 19870 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:47:09.820135 19870 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:47:09.821311 19870 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 03:47:09.821354 19870 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:47:09.821367 19870 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:47:09.821406 19870 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 03:47:09.821435 19870 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260806 03:47:09.821460 19870 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260806 03:47:09.821484 19870 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:37391/jwks.json
I20260806 03:47:09.822894 19870 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.253.212:39239
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785988004637995-12279-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785988004637995-12279-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.253.254:34835
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.253.254:34835
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:37391/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785988004637995-12279-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.253.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19870
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785988004637995-12279-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:47:09.823160 19870 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:47:09.823488 19870 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:47:09.828589 19879 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:47:09.828611 19884 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:47:09.828589 19882 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:47:09.828893 19870 server_base.cc:1034] running on GCE node
I20260806 03:47:09.829157 19870 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:47:09.829421 19870 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:47:09.830610 19870 hybrid_clock.cc:648] HybridClock initialized: now 1785988029830608 us; error 44 us; skew 500 ppm
I20260806 03:47:09.831950 19870 webserver.cc:460] Webserver started at http://127.11.253.254:43947/ using document root <none> and password file <none>
I20260806 03:47:09.832185 19870 fs_manager.cc:359] Metadata directory not provided
I20260806 03:47:09.832234 19870 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:47:09.832330 19870 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:47:09.833420 19870 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785988004637995-12279-0/minicluster-data/master-0/data/instance:
uuid: "0d2f3baa5ea44cfe8a36b6c9c4810270"
format_stamp: "Formatted at 2026-08-06 03:47:09 on dist-test-slave-6ntk"
server_key: "2c2743d013a11c5297f82a9152e4eaba"
server_key_iv: "a4c99edd99e2755c73d9bbf0f503d52b"
server_key_version: "encryptionkey@0"
I20260806 03:47:09.833802 19870 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785988004637995-12279-0/minicluster-data/master-0/wal/instance:
uuid: "0d2f3baa5ea44cfe8a36b6c9c4810270"
format_stamp: "Formatted at 2026-08-06 03:47:09 on dist-test-slave-6ntk"
server_key: "2c2743d013a11c5297f82a9152e4eaba"
server_key_iv: "a4c99edd99e2755c73d9bbf0f503d52b"
server_key_version: "encryptionkey@0"
I20260806 03:47:09.835213 19870 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 03:47:09.835996 19899 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:47:09.836175 19870 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 03:47:09.836282 19870 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785988004637995-12279-0/minicluster-data/master-0/data,/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785988004637995-12279-0/minicluster-data/master-0/wal
uuid: "0d2f3baa5ea44cfe8a36b6c9c4810270"
format_stamp: "Formatted at 2026-08-06 03:47:09 on dist-test-slave-6ntk"
server_key: "2c2743d013a11c5297f82a9152e4eaba"
server_key_iv: "a4c99edd99e2755c73d9bbf0f503d52b"
server_key_version: "encryptionkey@0"
I20260806 03:47:09.836366 19870 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785988004637995-12279-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785988004637995-12279-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785988004637995-12279-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:47:09.881448 19870 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:47:09.881857 19870 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:47:09.882064 19870 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:47:09.886196 19870 rpc_server.cc:307] RPC server started. Bound to: 127.11.253.254:34835
I20260806 03:47:09.886252 19967 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.253.254:34835 every 8 connection(s)
I20260806 03:47:09.886641 19870 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785988004637995-12279-0/minicluster-data/master-0/data/info.pb
I20260806 03:47:09.887368 19969 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:47:09.890239 19969 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0d2f3baa5ea44cfe8a36b6c9c4810270: Bootstrap starting.
I20260806 03:47:09.890630 12279 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskToVWRe/build/release/bin/kudu as pid 19870
I20260806 03:47:09.890766 12279 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785988004637995-12279-0/minicluster-data/master-0/wal/instance
I20260806 03:47:09.890986 12279 external_mini_cluster.cc:1428] Setting key 060d69fa398b3678bdd200bb78cec090
I20260806 03:47:09.891033 19969 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0d2f3baa5ea44cfe8a36b6c9c4810270: Neither blocks nor log segments found. Creating new log.
I20260806 03:47:09.891371 19969 log.cc:824] T 00000000000000000000000000000000 P 0d2f3baa5ea44cfe8a36b6c9c4810270: Log is configured to *not* fsync() on all Append() calls
I20260806 03:47:09.892148 19969 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0d2f3baa5ea44cfe8a36b6c9c4810270: No bootstrap required, opened a new log
I20260806 03:47:09.894405 19969 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0d2f3baa5ea44cfe8a36b6c9c4810270 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d2f3baa5ea44cfe8a36b6c9c4810270" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 34835 } }
I20260806 03:47:09.894572 19969 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0d2f3baa5ea44cfe8a36b6c9c4810270 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:47:09.894603 19969 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0d2f3baa5ea44cfe8a36b6c9c4810270 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d2f3baa5ea44cfe8a36b6c9c4810270, State: Initialized, Role: FOLLOWER
I20260806 03:47:09.894729 19969 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0d2f3baa5ea44cfe8a36b6c9c4810270 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d2f3baa5ea44cfe8a36b6c9c4810270" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 34835 } }
I20260806 03:47:09.894804 19969 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0d2f3baa5ea44cfe8a36b6c9c4810270 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 03:47:09.894840 19969 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0d2f3baa5ea44cfe8a36b6c9c4810270 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 03:47:09.894899 19969 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0d2f3baa5ea44cfe8a36b6c9c4810270 [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:47:09.895516 19969 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0d2f3baa5ea44cfe8a36b6c9c4810270 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d2f3baa5ea44cfe8a36b6c9c4810270" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 34835 } }
I20260806 03:47:09.895637 19969 leader_election.cc:302] T 00000000000000000000000000000000 P 0d2f3baa5ea44cfe8a36b6c9c4810270 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0d2f3baa5ea44cfe8a36b6c9c4810270; no voters:
I20260806 03:47:09.895803 19969 leader_election.cc:288] T 00000000000000000000000000000000 P 0d2f3baa5ea44cfe8a36b6c9c4810270 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 03:47:09.896035 19977 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0d2f3baa5ea44cfe8a36b6c9c4810270 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 03:47:09.896090 19969 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0d2f3baa5ea44cfe8a36b6c9c4810270 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 03:47:09.896354 19977 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0d2f3baa5ea44cfe8a36b6c9c4810270 [term 1 LEADER]: Becoming Leader. State: Replica: 0d2f3baa5ea44cfe8a36b6c9c4810270, State: Running, Role: LEADER
I20260806 03:47:09.896456 19977 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0d2f3baa5ea44cfe8a36b6c9c4810270 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d2f3baa5ea44cfe8a36b6c9c4810270" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 34835 } }
I20260806 03:47:09.896832 19976 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0d2f3baa5ea44cfe8a36b6c9c4810270 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0d2f3baa5ea44cfe8a36b6c9c4810270" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d2f3baa5ea44cfe8a36b6c9c4810270" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 34835 } } }
I20260806 03:47:09.896922 19976 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0d2f3baa5ea44cfe8a36b6c9c4810270 [sys.catalog]: This master's current role is: LEADER
I20260806 03:47:09.897037 19976 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0d2f3baa5ea44cfe8a36b6c9c4810270 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0d2f3baa5ea44cfe8a36b6c9c4810270. Latest consensus state: current_term: 1 leader_uuid: "0d2f3baa5ea44cfe8a36b6c9c4810270" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d2f3baa5ea44cfe8a36b6c9c4810270" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 34835 } } }
I20260806 03:47:09.897111 19976 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0d2f3baa5ea44cfe8a36b6c9c4810270 [sys.catalog]: This master's current role is: LEADER
I20260806 03:47:09.897289 19975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:09.891546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45232 (local address 127.11.253.254:34835)
0806 03:47:09.891766 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0806 03:47:09.891776 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 03:47:09.891810 (+ 34us) server_negotiation.cc:407] Connection header received
0806 03:47:09.892370 (+ 560us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:47:09.892378 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:47:09.892442 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:47:09.892567 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:47:09.892852 (+ 285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:09.893728 (+ 876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:09.894914 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:09.895402 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:09.895496 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:09.896271 (+ 775us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:47:09.896276 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:47:09.896276 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 03:47:09.896559 (+ 283us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:47:09.896607 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 03:47:09.896624 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:47:09.896722 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0806 03:47:09.896929 (+ 207us) server_negotiation.cc:299] Negotiation successful
0806 03:47:09.897016 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":92,"threads_started":1}
I20260806 03:47:09.897401 19990 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 03:47:09.897976 19990 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 03:47:09.898406 12279 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 03:47:09.899580 19990 catalog_manager.cc:1347] Generated new cluster ID: 71f557087b964c8183f90ed32d83d7f5
I20260806 03:47:09.899639 19990 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 03:47:09.908344 19990 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 03:47:09.908838 19990 catalog_manager.cc:1504] Loading token signing keys...
I20260806 03:47:09.918303 19990 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0d2f3baa5ea44cfe8a36b6c9c4810270: Generated new TSK 0
I20260806 03:47:09.918510 19990 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 03:47:09.932394 19975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:09.928859 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45248 (local address 127.11.253.254:34835)
0806 03:47:09.928940 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0806 03:47:09.928942 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:47:09.929033 (+ 91us) server_negotiation.cc:407] Connection header received
0806 03:47:09.929118 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:47:09.929119 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:47:09.929129 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:47:09.929189 (+ 60us) server_negotiation.cc:226] Negotiated authn=JWT
0806 03:47:09.929330 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:09.929829 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:09.930948 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:09.931140 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:09.931191 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:09.931366 (+ 175us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0806 03:47:09.932160 (+ 794us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37391/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0806 03:47:09.932308 (+ 148us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:45248: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37391/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":51}
W20260806 03:47:09.932473 19975 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:45248: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37391/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260806 03:47:09.932505 20008 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 03:47:09.928767 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:34835 (local address 127.0.0.1:45248)
0806 03:47:09.928964 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0806 03:47:09.928978 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 03:47:09.929046 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:47:09.929202 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:47:09.929203 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:47:09.929205 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0806 03:47:09.929269 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:09.929272 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:09.929935 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:09.929935 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:09.930852 (+ 917us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:09.930855 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:09.931210 (+ 355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:09.931211 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:09.931274 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:09.931279 (+ 5us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0806 03:47:09.932289 (+ 1010us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37391/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0806 03:47:09.932380 (+ 91us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.11.253.254:34835: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37391/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":144,"threads_started":1}
I20260806 03:47:09.933004 12279 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskToVWRe/build/release/bin/kudu with pid 19870
E20260806 03:47:09.948931 20009 webserver.cc:533] Webserver: sslize failed:
E20260806 03:47:09.949011 20009 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-08-06T03:47:09Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (254 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-06T03:47:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T03:47:09Z Disabled control of system clock
I20260806 03:47:09.988904 12279 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskToVWRe/build/release/bin/kudu
/tmp/dist-test-taskToVWRe/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785988004637995-12279-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785988004637995-12279-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785988004637995-12279-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785988004637995-12279-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.253.254:32917
--webserver_interface=127.11.253.254
--webserver_port=0
--builtin_ntp_servers=127.11.253.212:41009
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.253.254:32917
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785988004637995-12279-0/testchainca.cert with env {}
W20260806 03:47:10.073032 20015 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:47:10.073243 20015 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:47:10.073274 20015 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:47:10.074445 20015 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 03:47:10.074487 20015 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:47:10.074501 20015 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:47:10.074512 20015 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 03:47:10.074524 20015 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260806 03:47:10.074537 20015 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260806 03:47:10.074548 20015 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260806 03:47:10.075951 20015 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.253.212:41009
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785988004637995-12279-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785988004637995-12279-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.253.254:32917
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.253.254:32917
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785988004637995-12279-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.253.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785988004637995-12279-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20015
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785988004637995-12279-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:47:10.076143 20015 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:47:10.076314 20015 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:47:10.081067 20021 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:47:10.081076 20022 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:47:10.081084 20026 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:47:10.081450 20015 server_base.cc:1034] running on GCE node
I20260806 03:47:10.081692 20015 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:47:10.082027 20015 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:47:10.083171 20015 hybrid_clock.cc:648] HybridClock initialized: now 1785988030083160 us; error 32 us; skew 500 ppm
I20260806 03:47:10.084482 20015 webserver.cc:460] Webserver started at http://127.11.253.254:39309/ using document root <none> and password file <none>
I20260806 03:47:10.084717 20015 fs_manager.cc:359] Metadata directory not provided
I20260806 03:47:10.084762 20015 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:47:10.084909 20015 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:47:10.086007 20015 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785988004637995-12279-0/minicluster-data/master-0/data/instance:
uuid: "d59c66d66a124c2fabc802ebfd253dd1"
format_stamp: "Formatted at 2026-08-06 03:47:10 on dist-test-slave-6ntk"
server_key: "f355d4e32412107dc7ed90f382e48202"
server_key_iv: "78cebe8805125e4d562fc08c3c132725"
server_key_version: "encryptionkey@0"
I20260806 03:47:10.086366 20015 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785988004637995-12279-0/minicluster-data/master-0/wal/instance:
uuid: "d59c66d66a124c2fabc802ebfd253dd1"
format_stamp: "Formatted at 2026-08-06 03:47:10 on dist-test-slave-6ntk"
server_key: "f355d4e32412107dc7ed90f382e48202"
server_key_iv: "78cebe8805125e4d562fc08c3c132725"
server_key_version: "encryptionkey@0"
I20260806 03:47:10.087679 20015 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260806 03:47:10.088500 20034 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:47:10.088706 20015 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 03:47:10.088814 20015 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785988004637995-12279-0/minicluster-data/master-0/data,/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785988004637995-12279-0/minicluster-data/master-0/wal
uuid: "d59c66d66a124c2fabc802ebfd253dd1"
format_stamp: "Formatted at 2026-08-06 03:47:10 on dist-test-slave-6ntk"
server_key: "f355d4e32412107dc7ed90f382e48202"
server_key_iv: "78cebe8805125e4d562fc08c3c132725"
server_key_version: "encryptionkey@0"
I20260806 03:47:10.088887 20015 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785988004637995-12279-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785988004637995-12279-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785988004637995-12279-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:47:10.109592 20015 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:47:10.109989 20015 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:47:10.110160 20015 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:47:10.114925 20015 rpc_server.cc:307] RPC server started. Bound to: 127.11.253.254:32917
I20260806 03:47:10.114960 20115 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.253.254:32917 every 8 connection(s)
I20260806 03:47:10.115391 20015 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785988004637995-12279-0/minicluster-data/master-0/data/info.pb
I20260806 03:47:10.116437 20116 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:47:10.118821 20116 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d59c66d66a124c2fabc802ebfd253dd1: Bootstrap starting.
I20260806 03:47:10.119416 20116 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d59c66d66a124c2fabc802ebfd253dd1: Neither blocks nor log segments found. Creating new log.
I20260806 03:47:10.119702 20116 log.cc:824] T 00000000000000000000000000000000 P d59c66d66a124c2fabc802ebfd253dd1: Log is configured to *not* fsync() on all Append() calls
I20260806 03:47:10.120361 20116 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d59c66d66a124c2fabc802ebfd253dd1: No bootstrap required, opened a new log
I20260806 03:47:10.122066 20116 raft_consensus.cc:348] T 00000000000000000000000000000000 P d59c66d66a124c2fabc802ebfd253dd1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d59c66d66a124c2fabc802ebfd253dd1" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 32917 } }
I20260806 03:47:10.122203 20116 raft_consensus.cc:374] T 00000000000000000000000000000000 P d59c66d66a124c2fabc802ebfd253dd1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:47:10.122226 20116 raft_consensus.cc:729] T 00000000000000000000000000000000 P d59c66d66a124c2fabc802ebfd253dd1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d59c66d66a124c2fabc802ebfd253dd1, State: Initialized, Role: FOLLOWER
I20260806 03:47:10.122440 20116 consensus_queue.cc:260] T 00000000000000000000000000000000 P d59c66d66a124c2fabc802ebfd253dd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d59c66d66a124c2fabc802ebfd253dd1" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 32917 } }
I20260806 03:47:10.122507 20116 raft_consensus.cc:388] T 00000000000000000000000000000000 P d59c66d66a124c2fabc802ebfd253dd1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 03:47:10.122545 20116 raft_consensus.cc:482] T 00000000000000000000000000000000 P d59c66d66a124c2fabc802ebfd253dd1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 03:47:10.122596 20116 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d59c66d66a124c2fabc802ebfd253dd1 [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:47:10.123143 20116 raft_consensus.cc:504] T 00000000000000000000000000000000 P d59c66d66a124c2fabc802ebfd253dd1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d59c66d66a124c2fabc802ebfd253dd1" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 32917 } }
I20260806 03:47:10.123255 20116 leader_election.cc:302] T 00000000000000000000000000000000 P d59c66d66a124c2fabc802ebfd253dd1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d59c66d66a124c2fabc802ebfd253dd1; no voters:
I20260806 03:47:10.123456 20116 leader_election.cc:288] T 00000000000000000000000000000000 P d59c66d66a124c2fabc802ebfd253dd1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 03:47:10.123528 20120 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d59c66d66a124c2fabc802ebfd253dd1 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 03:47:10.123705 20120 raft_consensus.cc:686] T 00000000000000000000000000000000 P d59c66d66a124c2fabc802ebfd253dd1 [term 1 LEADER]: Becoming Leader. State: Replica: d59c66d66a124c2fabc802ebfd253dd1, State: Running, Role: LEADER
I20260806 03:47:10.123701 12279 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskToVWRe/build/release/bin/kudu as pid 20015
I20260806 03:47:10.123723 20116 sys_catalog.cc:564] T 00000000000000000000000000000000 P d59c66d66a124c2fabc802ebfd253dd1 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 03:47:10.123790 12279 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785988004637995-12279-0/minicluster-data/master-0/wal/instance
I20260806 03:47:10.123821 20120 consensus_queue.cc:237] T 00000000000000000000000000000000 P d59c66d66a124c2fabc802ebfd253dd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d59c66d66a124c2fabc802ebfd253dd1" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 32917 } }
I20260806 03:47:10.123988 12279 external_mini_cluster.cc:1428] Setting key d97ffec90e383a57edc7bad9a8cea828
I20260806 03:47:10.124168 20120 sys_catalog.cc:455] T 00000000000000000000000000000000 P d59c66d66a124c2fabc802ebfd253dd1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d59c66d66a124c2fabc802ebfd253dd1. Latest consensus state: current_term: 1 leader_uuid: "d59c66d66a124c2fabc802ebfd253dd1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d59c66d66a124c2fabc802ebfd253dd1" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 32917 } } }
I20260806 03:47:10.124250 20120 sys_catalog.cc:458] T 00000000000000000000000000000000 P d59c66d66a124c2fabc802ebfd253dd1 [sys.catalog]: This master's current role is: LEADER
I20260806 03:47:10.124392 20121 sys_catalog.cc:455] T 00000000000000000000000000000000 P d59c66d66a124c2fabc802ebfd253dd1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d59c66d66a124c2fabc802ebfd253dd1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d59c66d66a124c2fabc802ebfd253dd1" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 32917 } } }
I20260806 03:47:10.124526 20130 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 03:47:10.124534 20121 sys_catalog.cc:458] T 00000000000000000000000000000000 P d59c66d66a124c2fabc802ebfd253dd1 [sys.catalog]: This master's current role is: LEADER
I20260806 03:47:10.125036 20130 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 03:47:10.126945 20130 catalog_manager.cc:1347] Generated new cluster ID: 85e1fe188d844dd88401846425f02a70
I20260806 03:47:10.127002 20130 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 03:47:10.130627 20134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:10.124611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53168 (local address 127.11.253.254:32917)
0806 03:47:10.124828 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0806 03:47:10.124839 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 03:47:10.125291 (+ 452us) server_negotiation.cc:407] Connection header received
0806 03:47:10.125636 (+ 345us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:47:10.125645 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:47:10.125723 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:47:10.125862 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:47:10.126250 (+ 388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:10.127368 (+ 1118us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:10.128452 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:10.128930 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:10.129087 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:10.129844 (+ 757us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:47:10.129848 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:47:10.129848 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 03:47:10.130082 (+ 234us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:47:10.130117 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 03:47:10.130129 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:47:10.130231 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0806 03:47:10.130349 (+ 118us) server_negotiation.cc:299] Negotiation successful
0806 03:47:10.130440 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":97,"threads_started":1}
I20260806 03:47:10.131687 12279 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 03:47:10.141391 20130 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 03:47:10.141968 20130 catalog_manager.cc:1504] Loading token signing keys...
I20260806 03:47:10.150595 20130 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d59c66d66a124c2fabc802ebfd253dd1: Generated new TSK 0
I20260806 03:47:10.150805 20130 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 03:47:10.379443 20134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:10.162838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53184 (local address 127.11.253.254:32917)
0806 03:47:10.162899 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0806 03:47:10.162902 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 03:47:10.163007 (+ 105us) server_negotiation.cc:407] Connection header received
0806 03:47:10.163081 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:47:10.163082 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:47:10.163092 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:47:10.163150 (+ 58us) server_negotiation.cc:226] Negotiated authn=JWT
0806 03:47:10.163306 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:10.163798 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:10.164927 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:10.165111 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:10.165174 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:10.165351 (+ 177us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0806 03:47:10.379210 (+213859us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0806 03:47:10.379377 (+ 167us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:53184: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":34}
W20260806 03:47:10.379537 20134 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:53184: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260806 03:47:10.379613 20158 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 03:47:10.162777 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:32917 (local address 127.0.0.1:53184)
0806 03:47:10.162936 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0806 03:47:10.162948 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 03:47:10.163016 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:47:10.163167 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:47:10.163168 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:47:10.163170 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0806 03:47:10.163229 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:10.163231 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:10.163874 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:10.163874 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:10.164827 (+ 953us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:10.164830 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:10.165189 (+ 359us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:10.165190 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:10.165251 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:10.165257 (+ 6us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0806 03:47:10.379375 (+214118us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0806 03:47:10.379497 (+ 122us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.11.253.254:32917: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":107,"threads_started":1}
I20260806 03:47:10.380035 12279 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskToVWRe/build/release/bin/kudu with pid 20015
2026-08-06T03:47:10Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (424 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (94 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (96 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785988004637995-12279-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 03:47:10 dist-test-slave-6ntk krb5kdc[20177](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 03:47:10 dist-test-slave-6ntk krb5kdc[20177](info): set up 2 sockets
Aug 06 03:47:10 dist-test-slave-6ntk krb5kdc[20177](info): commencing operation
krb5kdc: starting...
W20260806 03:47:11.796976 12279 mini_kdc.cc:121] Time spent starting KDC: real 1.201s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785988004637995-12279-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785988004637995-12279-0/krb5kdc/test-user.keytab.
Aug 06 03:47:11 dist-test-slave-6ntk krb5kdc[20177](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988031, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T03:47:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T03:47:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.253.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.253.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.253.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.254.keytab.
Entry for principal kudu/127.11.253.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.254.keytab.
I20260806 03:47:11.925194 12279 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskToVWRe/build/release/bin/kudu
/tmp/dist-test-taskToVWRe/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785988004637995-12279-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785988004637995-12279-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785988004637995-12279-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785988004637995-12279-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.253.254:45231
--webserver_interface=127.11.253.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.254.keytab
--principal=kudu/127.11.253.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.253.212:36255
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.253.254:45231
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785988004637995-12279-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785988004637995-12279-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785988004637995-12279-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:47:12.010329 20196 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:47:12.010645 20196 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:47:12.010687 20196 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:47:12.012498 20196 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 03:47:12.012575 20196 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:47:12.012599 20196 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:47:12.012634 20196 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 03:47:12.012661 20196 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 03:47:12.014374 20196 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.253.212:36255
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785988004637995-12279-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785988004637995-12279-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.253.254:45231
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.254.keytab
--principal=kudu/127.11.253.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.253.254:45231
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785988004637995-12279-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.253.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20196
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785988004637995-12279-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:47:12.014678 20196 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:47:12.014935 20196 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:47:12.020120 20204 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:47:12.020172 20201 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:47:12.020390 20196 server_base.cc:1034] running on GCE node
W20260806 03:47:12.020121 20202 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:47:12.020780 20196 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:47:12.021059 20196 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:47:12.022209 20196 hybrid_clock.cc:648] HybridClock initialized: now 1785988032022185 us; error 38 us; skew 500 ppm
Aug 06 03:47:12 dist-test-slave-6ntk krb5kdc[20177](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988032, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.253.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:47:12.024842 20196 init.cc:377] Logged in from keytab as kudu/127.11.253.254@KRBTEST.COM (short username kudu)
I20260806 03:47:12.025317 20196 webserver.cc:460] Webserver started at http://127.11.253.254:32991/ using document root <none> and password file <none>
I20260806 03:47:12.025621 20196 fs_manager.cc:359] Metadata directory not provided
I20260806 03:47:12.025692 20196 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:47:12.025861 20196 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:47:12.027094 20196 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785988004637995-12279-0/minicluster-data/master-0/data/instance:
uuid: "a7268d9a888245188084ab940a3302b1"
format_stamp: "Formatted at 2026-08-06 03:47:12 on dist-test-slave-6ntk"
server_key: "542519380c0b335ee5aa0e080e4129e1"
server_key_iv: "448a10ebde9932cfa51fe3d643f2af3e"
server_key_version: "encryptionkey@0"
I20260806 03:47:12.027393 20196 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785988004637995-12279-0/minicluster-data/master-0/wal/instance:
uuid: "a7268d9a888245188084ab940a3302b1"
format_stamp: "Formatted at 2026-08-06 03:47:12 on dist-test-slave-6ntk"
server_key: "542519380c0b335ee5aa0e080e4129e1"
server_key_iv: "448a10ebde9932cfa51fe3d643f2af3e"
server_key_version: "encryptionkey@0"
I20260806 03:47:12.028673 20196 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 03:47:12.029356 20213 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:47:12.029537 20196 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 03:47:12.029613 20196 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785988004637995-12279-0/minicluster-data/master-0/data,/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785988004637995-12279-0/minicluster-data/master-0/wal
uuid: "a7268d9a888245188084ab940a3302b1"
format_stamp: "Formatted at 2026-08-06 03:47:12 on dist-test-slave-6ntk"
server_key: "542519380c0b335ee5aa0e080e4129e1"
server_key_iv: "448a10ebde9932cfa51fe3d643f2af3e"
server_key_version: "encryptionkey@0"
I20260806 03:47:12.029665 20196 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785988004637995-12279-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785988004637995-12279-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785988004637995-12279-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:47:12.043342 20196 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:47:12.045208 20196 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:47:12.045327 20196 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:47:12.049368 20196 rpc_server.cc:307] RPC server started. Bound to: 127.11.253.254:45231
I20260806 03:47:12.049430 20286 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.253.254:45231 every 8 connection(s)
I20260806 03:47:12.049798 20196 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785988004637995-12279-0/minicluster-data/master-0/data/info.pb
I20260806 03:47:12.050664 12279 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskToVWRe/build/release/bin/kudu as pid 20196
I20260806 03:47:12.050753 12279 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785988004637995-12279-0/minicluster-data/master-0/wal/instance
I20260806 03:47:12.050825 20287 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:47:12.050984 12279 external_mini_cluster.cc:1428] Setting key 7e0f331226211974cf802422246b03cb
I20260806 03:47:12.053644 20287 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a7268d9a888245188084ab940a3302b1: Bootstrap starting.
I20260806 03:47:12.054325 20287 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a7268d9a888245188084ab940a3302b1: Neither blocks nor log segments found. Creating new log.
I20260806 03:47:12.054608 20287 log.cc:824] T 00000000000000000000000000000000 P a7268d9a888245188084ab940a3302b1: Log is configured to *not* fsync() on all Append() calls
I20260806 03:47:12.055305 20287 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a7268d9a888245188084ab940a3302b1: No bootstrap required, opened a new log
Aug 06 03:47:12 dist-test-slave-6ntk krb5kdc[20177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988031, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.253.254@KRBTEST.COM
I20260806 03:47:12.057595 20287 raft_consensus.cc:348] T 00000000000000000000000000000000 P a7268d9a888245188084ab940a3302b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7268d9a888245188084ab940a3302b1" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 45231 } }
I20260806 03:47:12.057772 20287 raft_consensus.cc:374] T 00000000000000000000000000000000 P a7268d9a888245188084ab940a3302b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:47:12.057832 20287 raft_consensus.cc:729] T 00000000000000000000000000000000 P a7268d9a888245188084ab940a3302b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a7268d9a888245188084ab940a3302b1, State: Initialized, Role: FOLLOWER
I20260806 03:47:12.058019 20287 consensus_queue.cc:260] T 00000000000000000000000000000000 P a7268d9a888245188084ab940a3302b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7268d9a888245188084ab940a3302b1" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 45231 } }
I20260806 03:47:12.058087 20287 raft_consensus.cc:388] T 00000000000000000000000000000000 P a7268d9a888245188084ab940a3302b1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 03:47:12.058128 20287 raft_consensus.cc:482] T 00000000000000000000000000000000 P a7268d9a888245188084ab940a3302b1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 03:47:12.058180 20287 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a7268d9a888245188084ab940a3302b1 [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:47:12.058873 20287 raft_consensus.cc:504] T 00000000000000000000000000000000 P a7268d9a888245188084ab940a3302b1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7268d9a888245188084ab940a3302b1" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 45231 } }
I20260806 03:47:12.058995 20287 leader_election.cc:302] T 00000000000000000000000000000000 P a7268d9a888245188084ab940a3302b1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a7268d9a888245188084ab940a3302b1; no voters:
I20260806 03:47:12.059192 20287 leader_election.cc:288] T 00000000000000000000000000000000 P a7268d9a888245188084ab940a3302b1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 03:47:12.059259 20296 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a7268d9a888245188084ab940a3302b1 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 03:47:12.059422 20296 raft_consensus.cc:686] T 00000000000000000000000000000000 P a7268d9a888245188084ab940a3302b1 [term 1 LEADER]: Becoming Leader. State: Replica: a7268d9a888245188084ab940a3302b1, State: Running, Role: LEADER
I20260806 03:47:12.059504 20287 sys_catalog.cc:564] T 00000000000000000000000000000000 P a7268d9a888245188084ab940a3302b1 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 03:47:12.059516 20296 consensus_queue.cc:237] T 00000000000000000000000000000000 P a7268d9a888245188084ab940a3302b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7268d9a888245188084ab940a3302b1" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 45231 } }
I20260806 03:47:12.060695 20297 sys_catalog.cc:455] T 00000000000000000000000000000000 P a7268d9a888245188084ab940a3302b1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a7268d9a888245188084ab940a3302b1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7268d9a888245188084ab940a3302b1" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 45231 } } }
I20260806 03:47:12.060794 20297 sys_catalog.cc:458] T 00000000000000000000000000000000 P a7268d9a888245188084ab940a3302b1 [sys.catalog]: This master's current role is: LEADER
I20260806 03:47:12.061095 20298 sys_catalog.cc:455] T 00000000000000000000000000000000 P a7268d9a888245188084ab940a3302b1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a7268d9a888245188084ab940a3302b1. Latest consensus state: current_term: 1 leader_uuid: "a7268d9a888245188084ab940a3302b1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7268d9a888245188084ab940a3302b1" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 45231 } } }
I20260806 03:47:12.061292 20298 sys_catalog.cc:458] T 00000000000000000000000000000000 P a7268d9a888245188084ab940a3302b1 [sys.catalog]: This master's current role is: LEADER
W20260806 03:47:12.061394 20326 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a7268d9a888245188084ab940a3302b1: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 03:47:12.061453 20326 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 03:47:12.062444 20293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:12.051535 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43888 (local address 127.11.253.254:45231)
0806 03:47:12.051745 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0806 03:47:12.051753 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 03:47:12.051778 (+ 25us) server_negotiation.cc:407] Connection header received
0806 03:47:12.052245 (+ 467us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:47:12.052252 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:47:12.052298 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:47:12.052449 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:47:12.053096 (+ 647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:12.054092 (+ 996us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:12.054958 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:12.055359 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:12.055485 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:12.058265 (+ 2780us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:47:12.058270 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:47:12.058271 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:47:12.058284 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:47:12.060637 (+ 2353us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:47:12.061081 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:47:12.061084 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:47:12.061084 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:47:12.061209 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:47:12.061470 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:47:12.061471 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:47:12.061471 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:47:12.061775 (+ 304us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:47:12.061953 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0806 03:47:12.062095 (+ 142us) server_negotiation.cc:299] Negotiation successful
0806 03:47:12.062234 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":166,"thread_start_us":93,"threads_started":1}
I20260806 03:47:12.065994 20328 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 03:47:12.066177 20328 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 03:47:12.067724 20328 catalog_manager.cc:1347] Generated new cluster ID: c0cd1258012c484ea821b513caee06a3
I20260806 03:47:12.067781 20328 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 03:47:12.075600 20328 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 03:47:12.076346 20328 catalog_manager.cc:1504] Loading token signing keys...
I20260806 03:47:12.079924 20328 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a7268d9a888245188084ab940a3302b1: Generated new TSK 0
I20260806 03:47:12.080111 20328 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.253.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.253.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.253.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.193.keytab.
Entry for principal kudu/127.11.253.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.193.keytab.
I20260806 03:47:12.115130 12279 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskToVWRe/build/release/bin/kudu
/tmp/dist-test-taskToVWRe/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785988004637995-12279-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785988004637995-12279-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785988004637995-12279-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785988004637995-12279-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.253.193:0
--local_ip_for_outbound_sockets=127.11.253.193
--webserver_interface=127.11.253.193
--webserver_port=0
--tserver_master_addrs=127.11.253.254:45231
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.193.keytab
--principal=kudu/127.11.253.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.253.212:36255
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785988004637995-12279-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785988004637995-12279-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785988004637995-12279-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:47:12.210882 20341 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:47:12.211104 20341 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:47:12.211129 20341 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:47:12.212399 20341 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:47:12.212442 20341 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:47:12.212471 20341 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.253.193
I20260806 03:47:12.214032 20341 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.253.212:36255
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785988004637995-12279-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785988004637995-12279-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.193.keytab
--principal=kudu/127.11.253.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.253.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785988004637995-12279-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.253.193
--webserver_port=0
--tserver_master_addrs=127.11.253.254:45231
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20341
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.253.193
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785988004637995-12279-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:47:12.214282 20341 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:47:12.214514 20341 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:47:12.219918 20349 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:47:12.219904 20351 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:47:12.219933 20341 server_base.cc:1034] running on GCE node
W20260806 03:47:12.219904 20348 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:47:12.220407 20341 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:47:12.220681 20341 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:47:12.221850 20341 hybrid_clock.cc:648] HybridClock initialized: now 1785988032221850 us; error 41 us; skew 500 ppm
Aug 06 03:47:12 dist-test-slave-6ntk krb5kdc[20177](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988032, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.253.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:47:12.224581 20341 init.cc:377] Logged in from keytab as kudu/127.11.253.193@KRBTEST.COM (short username kudu)
I20260806 03:47:12.225067 20341 webserver.cc:460] Webserver started at http://127.11.253.193:33519/ using document root <none> and password file <none>
I20260806 03:47:12.225317 20341 fs_manager.cc:359] Metadata directory not provided
I20260806 03:47:12.225382 20341 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:47:12.225550 20341 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:47:12.226720 20341 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785988004637995-12279-0/minicluster-data/ts-0/data/instance:
uuid: "5bbb90de155344ddb38939f3b870ea04"
format_stamp: "Formatted at 2026-08-06 03:47:12 on dist-test-slave-6ntk"
server_key: "c78929216772068134592b75aa3bd439"
server_key_iv: "2948fb01dfde42d55f134c93ca7eb46e"
server_key_version: "encryptionkey@0"
I20260806 03:47:12.227087 20341 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785988004637995-12279-0/minicluster-data/ts-0/wal/instance:
uuid: "5bbb90de155344ddb38939f3b870ea04"
format_stamp: "Formatted at 2026-08-06 03:47:12 on dist-test-slave-6ntk"
server_key: "c78929216772068134592b75aa3bd439"
server_key_iv: "2948fb01dfde42d55f134c93ca7eb46e"
server_key_version: "encryptionkey@0"
I20260806 03:47:12.228432 20341 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260806 03:47:12.229127 20362 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:47:12.229300 20341 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 03:47:12.229400 20341 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785988004637995-12279-0/minicluster-data/ts-0/data,/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785988004637995-12279-0/minicluster-data/ts-0/wal
uuid: "5bbb90de155344ddb38939f3b870ea04"
format_stamp: "Formatted at 2026-08-06 03:47:12 on dist-test-slave-6ntk"
server_key: "c78929216772068134592b75aa3bd439"
server_key_iv: "2948fb01dfde42d55f134c93ca7eb46e"
server_key_version: "encryptionkey@0"
I20260806 03:47:12.229476 20341 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785988004637995-12279-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785988004637995-12279-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785988004637995-12279-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:47:12.247274 20341 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:47:12.249671 20341 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:47:12.249837 20341 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:47:12.250141 20341 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 03:47:12.250473 20341 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 03:47:12.250524 20341 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 03:47:12.250566 20341 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 03:47:12.250597 20341 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 03:47:12.257344 20341 rpc_server.cc:307] RPC server started. Bound to: 127.11.253.193:45713
I20260806 03:47:12.257402 20527 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.253.193:45713 every 8 connection(s)
I20260806 03:47:12.257743 20341 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785988004637995-12279-0/minicluster-data/ts-0/data/info.pb
I20260806 03:47:12.263168 12279 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskToVWRe/build/release/bin/kudu as pid 20341
I20260806 03:47:12.263346 12279 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785988004637995-12279-0/minicluster-data/ts-0/wal/instance
I20260806 03:47:12.263576 12279 external_mini_cluster.cc:1428] Setting key eda3030b4d582cab1e73015f8011fe13
Aug 06 03:47:12 dist-test-slave-6ntk krb5kdc[20177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988032, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.253.193@KRBTEST.COM for kudu/127.11.253.254@KRBTEST.COM
I20260806 03:47:12.268303 20293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:12.258555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.253.193:40945 (local address 127.11.253.254:45231)
0806 03:47:12.258639 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0806 03:47:12.258642 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 03:47:12.258836 (+ 194us) server_negotiation.cc:407] Connection header received
0806 03:47:12.259890 (+ 1054us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:47:12.259892 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:47:12.259903 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:47:12.260045 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:47:12.260818 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:12.261417 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:12.262431 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:12.262637 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:12.262697 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:12.265162 (+ 2465us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:47:12.265164 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:47:12.265164 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:47:12.265171 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:47:12.266858 (+ 1687us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:47:12.267272 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:47:12.267273 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:47:12.267274 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:47:12.267351 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:47:12.267586 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:47:12.267586 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:47:12.267587 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:47:12.267737 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:47:12.267808 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0806 03:47:12.268056 (+ 248us) server_negotiation.cc:299] Negotiation successful
0806 03:47:12.268158 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260806 03:47:12.268411 20531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:12.258527 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:45231 (local address 127.11.253.193:40945)
0806 03:47:12.258700 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0806 03:47:12.258745 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 03:47:12.259768 (+ 1023us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:47:12.260170 (+ 402us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:47:12.260186 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:47:12.260452 (+ 266us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:47:12.260721 (+ 269us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:12.260725 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:12.261508 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:12.261509 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:12.262354 (+ 845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:12.262357 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:12.262714 (+ 357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:12.262715 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:12.262819 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:12.263449 (+ 630us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:47:12.263460 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:47:12.265033 (+ 1573us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:47:12.266957 (+ 1924us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:47:12.266965 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:47:12.266967 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:47:12.267200 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:47:12.267420 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:47:12.267421 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:47:12.267421 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:47:12.267522 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:47:12.267802 (+ 280us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:47:12.267807 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:47:12.267924 (+ 117us) client_negotiation.cc:769] Sending connection context
0806 03:47:12.268010 (+ 86us) client_negotiation.cc:240] Negotiation successful
0806 03:47:12.268137 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":91,"threads_started":1}
I20260806 03:47:12.268960 20528 heartbeater.cc:347] Connected to a master server at 127.11.253.254:45231
I20260806 03:47:12.269060 20528 heartbeater.cc:464] Registering TS with master...
I20260806 03:47:12.269281 20528 heartbeater.cc:511] Master 127.11.253.254:45231 requested a full tablet report, sending...
I20260806 03:47:12.269891 20236 ts_manager.cc:194] Registered new tserver with Master: 5bbb90de155344ddb38939f3b870ea04 (127.11.253.193:45713)
I20260806 03:47:12.270881 20236 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.253.193@KRBTEST.COM'} at 127.11.253.193:40945
WARNING: no policy specified for kudu/127.11.253.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.253.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.253.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.194.keytab.
Entry for principal kudu/127.11.253.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.194.keytab.
I20260806 03:47:12.292840 12279 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskToVWRe/build/release/bin/kudu
/tmp/dist-test-taskToVWRe/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785988004637995-12279-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785988004637995-12279-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785988004637995-12279-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785988004637995-12279-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.253.194:0
--local_ip_for_outbound_sockets=127.11.253.194
--webserver_interface=127.11.253.194
--webserver_port=0
--tserver_master_addrs=127.11.253.254:45231
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.194.keytab
--principal=kudu/127.11.253.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.253.212:36255
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785988004637995-12279-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785988004637995-12279-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785988004637995-12279-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:47:12.384018 20535 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:47:12.384271 20535 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:47:12.384297 20535 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:47:12.385485 20535 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:47:12.385517 20535 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:47:12.385542 20535 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.253.194
I20260806 03:47:12.386996 20535 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.253.212:36255
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785988004637995-12279-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785988004637995-12279-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.194.keytab
--principal=kudu/127.11.253.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.253.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785988004637995-12279-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.253.194
--webserver_port=0
--tserver_master_addrs=127.11.253.254:45231
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20535
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.253.194
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785988004637995-12279-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:47:12.387181 20535 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:47:12.387362 20535 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:47:12.392547 20542 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:47:12.392551 20544 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:47:12.392668 20535 server_base.cc:1034] running on GCE node
W20260806 03:47:12.392551 20541 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:47:12.393095 20535 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:47:12.393306 20535 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:47:12.394444 20535 hybrid_clock.cc:648] HybridClock initialized: now 1785988032394431 us; error 32 us; skew 500 ppm
Aug 06 03:47:12 dist-test-slave-6ntk krb5kdc[20177](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988032, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.253.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:47:12.397024 20535 init.cc:377] Logged in from keytab as kudu/127.11.253.194@KRBTEST.COM (short username kudu)
I20260806 03:47:12.397420 20535 webserver.cc:460] Webserver started at http://127.11.253.194:46355/ using document root <none> and password file <none>
I20260806 03:47:12.397605 20535 fs_manager.cc:359] Metadata directory not provided
I20260806 03:47:12.397645 20535 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:47:12.397737 20535 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:47:12.398800 20535 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785988004637995-12279-0/minicluster-data/ts-1/data/instance:
uuid: "51938a48517b4d77ba9b65aa955915eb"
format_stamp: "Formatted at 2026-08-06 03:47:12 on dist-test-slave-6ntk"
server_key: "7aa4bc5c12ce875028b6e6487275c9fd"
server_key_iv: "49d8967f65aaa20e1e02c34e304b49c5"
server_key_version: "encryptionkey@0"
I20260806 03:47:12.399103 20535 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785988004637995-12279-0/minicluster-data/ts-1/wal/instance:
uuid: "51938a48517b4d77ba9b65aa955915eb"
format_stamp: "Formatted at 2026-08-06 03:47:12 on dist-test-slave-6ntk"
server_key: "7aa4bc5c12ce875028b6e6487275c9fd"
server_key_iv: "49d8967f65aaa20e1e02c34e304b49c5"
server_key_version: "encryptionkey@0"
I20260806 03:47:12.400396 20535 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 03:47:12.401073 20555 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:47:12.401304 20535 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 03:47:12.401417 20535 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785988004637995-12279-0/minicluster-data/ts-1/data,/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785988004637995-12279-0/minicluster-data/ts-1/wal
uuid: "51938a48517b4d77ba9b65aa955915eb"
format_stamp: "Formatted at 2026-08-06 03:47:12 on dist-test-slave-6ntk"
server_key: "7aa4bc5c12ce875028b6e6487275c9fd"
server_key_iv: "49d8967f65aaa20e1e02c34e304b49c5"
server_key_version: "encryptionkey@0"
I20260806 03:47:12.401484 20535 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785988004637995-12279-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785988004637995-12279-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785988004637995-12279-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:47:12.431048 20535 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:47:12.433436 20535 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:47:12.433637 20535 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:47:12.433911 20535 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 03:47:12.434245 20535 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 03:47:12.434279 20535 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 03:47:12.434301 20535 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 03:47:12.434352 20535 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 03:47:12.440275 20535 rpc_server.cc:307] RPC server started. Bound to: 127.11.253.194:40095
I20260806 03:47:12.440328 20721 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.253.194:40095 every 8 connection(s)
I20260806 03:47:12.440639 20535 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785988004637995-12279-0/minicluster-data/ts-1/data/info.pb
Aug 06 03:47:12 dist-test-slave-6ntk krb5kdc[20177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988032, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.253.194@KRBTEST.COM for kudu/127.11.253.254@KRBTEST.COM
I20260806 03:47:12.449353 20293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:12.441365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.253.194:40083 (local address 127.11.253.254:45231)
0806 03:47:12.441414 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0806 03:47:12.441417 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 03:47:12.441645 (+ 228us) server_negotiation.cc:407] Connection header received
0806 03:47:12.442586 (+ 941us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:47:12.442587 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:47:12.442598 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:47:12.442642 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:47:12.443263 (+ 621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:12.443749 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:12.444708 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:12.444890 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:12.444941 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:12.447012 (+ 2071us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:47:12.447013 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:47:12.447013 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:47:12.447020 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:47:12.448174 (+ 1154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:47:12.448543 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:47:12.448543 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:47:12.448544 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:47:12.448597 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:47:12.448806 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:47:12.448806 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:47:12.448806 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:47:12.448907 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:47:12.448955 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0806 03:47:12.449212 (+ 257us) server_negotiation.cc:299] Negotiation successful
0806 03:47:12.449278 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 03:47:12.449374 12279 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskToVWRe/build/release/bin/kudu as pid 20535
I20260806 03:47:12.449479 12279 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785988004637995-12279-0/minicluster-data/ts-1/wal/instance
I20260806 03:47:12.449632 20724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:12.441355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:45231 (local address 127.11.253.194:40083)
0806 03:47:12.441522 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0806 03:47:12.441560 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0806 03:47:12.442486 (+ 926us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:47:12.442727 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:47:12.442730 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:47:12.442984 (+ 254us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:47:12.443193 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:12.443197 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:12.443820 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:12.443821 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:12.444635 (+ 814us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:12.444637 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:12.444956 (+ 319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:12.444957 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:12.445055 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:12.445563 (+ 508us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:47:12.445571 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:47:12.446928 (+ 1357us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:47:12.448243 (+ 1315us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:47:12.448249 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:47:12.448251 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:47:12.448472 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:47:12.448651 (+ 179us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:47:12.448652 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:47:12.448652 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:47:12.448748 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:47:12.448957 (+ 209us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:47:12.448961 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:47:12.449072 (+ 111us) client_negotiation.cc:769] Sending connection context
0806 03:47:12.449196 (+ 124us) client_negotiation.cc:240] Negotiation successful
0806 03:47:12.449375 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":86,"threads_started":1}
I20260806 03:47:12.449674 12279 external_mini_cluster.cc:1428] Setting key 508e967638e4ad7a029ccc62585fe3d7
I20260806 03:47:12.450182 20722 heartbeater.cc:347] Connected to a master server at 127.11.253.254:45231
I20260806 03:47:12.450291 20722 heartbeater.cc:464] Registering TS with master...
I20260806 03:47:12.450469 20722 heartbeater.cc:511] Master 127.11.253.254:45231 requested a full tablet report, sending...
I20260806 03:47:12.450865 20233 ts_manager.cc:194] Registered new tserver with Master: 51938a48517b4d77ba9b65aa955915eb (127.11.253.194:40095)
I20260806 03:47:12.451367 20233 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.253.194@KRBTEST.COM'} at 127.11.253.194:40083
WARNING: no policy specified for kudu/127.11.253.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.253.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.253.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.195.keytab.
Entry for principal kudu/127.11.253.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.195.keytab.
I20260806 03:47:12.477232 12279 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskToVWRe/build/release/bin/kudu
/tmp/dist-test-taskToVWRe/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785988004637995-12279-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785988004637995-12279-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785988004637995-12279-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785988004637995-12279-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.253.195:0
--local_ip_for_outbound_sockets=127.11.253.195
--webserver_interface=127.11.253.195
--webserver_port=0
--tserver_master_addrs=127.11.253.254:45231
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.195.keytab
--principal=kudu/127.11.253.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.253.212:36255
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785988004637995-12279-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785988004637995-12279-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785988004637995-12279-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:47:12.565140 20729 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:47:12.565480 20729 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:47:12.565584 20729 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:47:12.566866 20729 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:47:12.566932 20729 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:47:12.566978 20729 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.253.195
I20260806 03:47:12.568501 20729 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.253.212:36255
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785988004637995-12279-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785988004637995-12279-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.195.keytab
--principal=kudu/127.11.253.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.253.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785988004637995-12279-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.253.195
--webserver_port=0
--tserver_master_addrs=127.11.253.254:45231
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20729
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.253.195
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785988004637995-12279-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:47:12.568779 20729 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:47:12.569018 20729 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 03:47:12.574074 20729 server_base.cc:1034] running on GCE node
W20260806 03:47:12.574021 20738 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:47:12.574043 20735 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:47:12.574021 20736 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:47:12.574486 20729 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:47:12.574725 20729 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:47:12.575865 20729 hybrid_clock.cc:648] HybridClock initialized: now 1785988032575847 us; error 40 us; skew 500 ppm
Aug 06 03:47:12 dist-test-slave-6ntk krb5kdc[20177](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988032, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.253.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:47:12.582321 20729 init.cc:377] Logged in from keytab as kudu/127.11.253.195@KRBTEST.COM (short username kudu)
I20260806 03:47:12.582978 20729 webserver.cc:460] Webserver started at http://127.11.253.195:42779/ using document root <none> and password file <none>
I20260806 03:47:12.583292 20729 fs_manager.cc:359] Metadata directory not provided
I20260806 03:47:12.583348 20729 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:47:12.583521 20729 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:47:12.584582 20729 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785988004637995-12279-0/minicluster-data/ts-2/data/instance:
uuid: "73d8cfbf2f254345abb3ef6e938d83bf"
format_stamp: "Formatted at 2026-08-06 03:47:12 on dist-test-slave-6ntk"
server_key: "1497195a3961547be72d5b502c388ffb"
server_key_iv: "8751d0047d5b2e0ca50d10c4ffc20088"
server_key_version: "encryptionkey@0"
I20260806 03:47:12.584946 20729 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785988004637995-12279-0/minicluster-data/ts-2/wal/instance:
uuid: "73d8cfbf2f254345abb3ef6e938d83bf"
format_stamp: "Formatted at 2026-08-06 03:47:12 on dist-test-slave-6ntk"
server_key: "1497195a3961547be72d5b502c388ffb"
server_key_iv: "8751d0047d5b2e0ca50d10c4ffc20088"
server_key_version: "encryptionkey@0"
I20260806 03:47:12.586365 20729 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 03:47:12.587201 20746 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:47:12.587404 20729 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 03:47:12.587502 20729 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785988004637995-12279-0/minicluster-data/ts-2/data,/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785988004637995-12279-0/minicluster-data/ts-2/wal
uuid: "73d8cfbf2f254345abb3ef6e938d83bf"
format_stamp: "Formatted at 2026-08-06 03:47:12 on dist-test-slave-6ntk"
server_key: "1497195a3961547be72d5b502c388ffb"
server_key_iv: "8751d0047d5b2e0ca50d10c4ffc20088"
server_key_version: "encryptionkey@0"
I20260806 03:47:12.587572 20729 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785988004637995-12279-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785988004637995-12279-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785988004637995-12279-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:47:12.604245 20729 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:47:12.606326 20729 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:47:12.606487 20729 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:47:12.606717 20729 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 03:47:12.607038 20729 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 03:47:12.607095 20729 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 03:47:12.607131 20729 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 03:47:12.607158 20729 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 03:47:12.613281 20729 rpc_server.cc:307] RPC server started. Bound to: 127.11.253.195:45635
I20260806 03:47:12.613328 20908 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.253.195:45635 every 8 connection(s)
I20260806 03:47:12.613658 20729 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785988004637995-12279-0/minicluster-data/ts-2/data/info.pb
Aug 06 03:47:12 dist-test-slave-6ntk krb5kdc[20177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988032, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.253.195@KRBTEST.COM for kudu/127.11.253.254@KRBTEST.COM
I20260806 03:47:12.622668 12279 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskToVWRe/build/release/bin/kudu as pid 20729
I20260806 03:47:12.622748 12279 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785988004637995-12279-0/minicluster-data/ts-2/wal/instance
I20260806 03:47:12.622972 12279 external_mini_cluster.cc:1428] Setting key 3ebd3370134b7e51cd07717a0612a5d1
I20260806 03:47:12.624107 20293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:12.614536 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.253.195:41475 (local address 127.11.253.254:45231)
0806 03:47:12.614610 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0806 03:47:12.614612 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:47:12.614881 (+ 269us) server_negotiation.cc:407] Connection header received
0806 03:47:12.615825 (+ 944us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:47:12.615826 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:47:12.615837 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:47:12.615891 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:47:12.616543 (+ 652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:12.617138 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:12.618351 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:12.618551 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:12.618618 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:12.621014 (+ 2396us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:47:12.621016 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:47:12.621017 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:47:12.621024 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:47:12.622397 (+ 1373us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:47:12.622915 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:47:12.622917 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:47:12.622918 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:47:12.622988 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:47:12.623329 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:47:12.623330 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:47:12.623331 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:47:12.623502 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:47:12.623571 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0806 03:47:12.623927 (+ 356us) server_negotiation.cc:299] Negotiation successful
0806 03:47:12.624011 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260806 03:47:12.624364 20914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:12.614503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:45231 (local address 127.11.253.195:41475)
0806 03:47:12.614738 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0806 03:47:12.614778 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 03:47:12.615714 (+ 936us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:47:12.615957 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:47:12.615961 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:47:12.616229 (+ 268us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:47:12.616435 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:12.616439 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:12.617307 (+ 868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:12.617308 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:12.618226 (+ 918us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:12.618229 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:12.618659 (+ 430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:12.618660 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:12.618773 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:12.619357 (+ 584us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:47:12.619366 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:47:12.620917 (+ 1551us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:47:12.622519 (+ 1602us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:47:12.622526 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:47:12.622528 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:47:12.622846 (+ 318us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:47:12.623088 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:47:12.623089 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:47:12.623090 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:47:12.623255 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:47:12.623571 (+ 316us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:47:12.623582 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:47:12.623771 (+ 189us) client_negotiation.cc:769] Sending connection context
0806 03:47:12.623890 (+ 119us) client_negotiation.cc:240] Negotiation successful
0806 03:47:12.624062 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":109,"threads_started":1}
I20260806 03:47:12.624985 20910 heartbeater.cc:347] Connected to a master server at 127.11.253.254:45231
I20260806 03:47:12.625077 20910 heartbeater.cc:464] Registering TS with master...
I20260806 03:47:12.625226 20910 heartbeater.cc:511] Master 127.11.253.254:45231 requested a full tablet report, sending...
I20260806 03:47:12.625623 20233 ts_manager.cc:194] Registered new tserver with Master: 73d8cfbf2f254345abb3ef6e938d83bf (127.11.253.195:45635)
I20260806 03:47:12.626042 20233 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.253.195@KRBTEST.COM'} at 127.11.253.195:41475
I20260806 03:47:12.634793 12279 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785988004637995-12279-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785988004637995-12279-0/krb5kdc/test-admin.keytab.
Aug 06 03:47:12 dist-test-slave-6ntk krb5kdc[20177](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988032, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:47:12.652078 12279 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260806 03:47:12.654003 20922 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 03:47:12.653123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52288 (local address 127.11.253.193:45713)
0806 03:47:12.653312 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0806 03:47:12.653318 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 03:47:12.653329 (+ 11us) server_negotiation.cc:407] Connection header received
0806 03:47:12.653362 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:47:12.653364 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:47:12.653378 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:47:12.653462 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:47:12.653865 (+ 403us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52288: BlockingRecv error: recv got EOF from 127.0.0.1:52288 (error 108)
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":82,"threads_started":1}
W20260806 03:47:12.654114 12279 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785988004637995-12279-0/krb5kdc/krb5cc' of type FILE
Aug 06 03:47:12 dist-test-slave-6ntk krb5kdc[20177](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988032, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:47:12.655236 12279 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 06 03:47:12 dist-test-slave-6ntk krb5kdc[20177](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988032, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:47:12.656231 12279 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 06 03:47:12 dist-test-slave-6ntk krb5kdc[20177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988032, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.253.193@KRBTEST.COM
I20260806 03:47:12.664697 20922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:12.656788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52300 (local address 127.11.253.193:45713)
0806 03:47:12.656855 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0806 03:47:12.656857 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:47:12.656866 (+ 9us) server_negotiation.cc:407] Connection header received
0806 03:47:12.656956 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:47:12.656957 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:47:12.656968 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:47:12.657025 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:47:12.657440 (+ 415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:12.658221 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:12.659171 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:12.659594 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:12.659677 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:12.662056 (+ 2379us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:47:12.662065 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:47:12.662066 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:47:12.662073 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:47:12.663767 (+ 1694us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:47:12.664071 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:47:12.664074 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:47:12.664075 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:47:12.664130 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:47:12.664275 (+ 145us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:47:12.664276 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:47:12.664276 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:47:12.664405 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:47:12.664519 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0806 03:47:12.664556 (+ 37us) server_negotiation.cc:299] Negotiation successful
0806 03:47:12.664619 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
W20260806 03:47:12.666702 20922 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 03:47:12.666072 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52306 (local address 127.11.253.193:45713)
0806 03:47:12.666121 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0806 03:47:12.666122 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 03:47:12.666173 (+ 51us) server_negotiation.cc:407] Connection header received
0806 03:47:12.666205 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:47:12.666206 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:47:12.666215 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:47:12.666278 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:47:12.666631 (+ 353us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52306: BlockingRecv error: recv got EOF from 127.0.0.1:52306 (error 108)
Metrics: {"server-negotiator.queue_time_us":29}
W20260806 03:47:12.666810 12279 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785988004637995-12279-0/krb5kdc/krb5cc' of type FILE
Aug 06 03:47:12 dist-test-slave-6ntk krb5kdc[20177](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988032, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:47:12.667781 12279 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 06 03:47:12 dist-test-slave-6ntk krb5kdc[20177](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988032, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:47:12.668679 12279 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 06 03:47:12 dist-test-slave-6ntk krb5kdc[20177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988032, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.253.193@KRBTEST.COM
I20260806 03:47:12.675449 20922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:12.669300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52310 (local address 127.11.253.193:45713)
0806 03:47:12.669367 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0806 03:47:12.669369 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:47:12.669376 (+ 7us) server_negotiation.cc:407] Connection header received
0806 03:47:12.669392 (+ 16us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:47:12.669393 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:47:12.669401 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:47:12.669447 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:47:12.669797 (+ 350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:12.670312 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:12.670995 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:12.671181 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:12.671230 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:12.673230 (+ 2000us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:47:12.673231 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:47:12.673231 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:47:12.673238 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:47:12.674476 (+ 1238us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:47:12.674806 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:47:12.674806 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:47:12.674807 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:47:12.674858 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:47:12.675012 (+ 154us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:47:12.675013 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:47:12.675013 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:47:12.675095 (+ 82us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:47:12.675164 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0806 03:47:12.675243 (+ 79us) server_negotiation.cc:299] Negotiation successful
0806 03:47:12.675299 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
W20260806 03:47:12.676971 20922 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 03:47:12.676432 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52322 (local address 127.11.253.193:45713)
0806 03:47:12.676483 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0806 03:47:12.676485 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:47:12.676563 (+ 78us) server_negotiation.cc:407] Connection header received
0806 03:47:12.676599 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:47:12.676599 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:47:12.676606 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:47:12.676651 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:47:12.676911 (+ 260us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52322: BlockingRecv error: recv got EOF from 127.0.0.1:52322 (error 108)
Metrics: {"server-negotiator.queue_time_us":30}
W20260806 03:47:12.677016 12279 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785988004637995-12279-0/krb5kdc/krb5cc' of type FILE
Aug 06 03:47:12 dist-test-slave-6ntk krb5kdc[20177](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988032, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:47:12.677953 12279 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 06 03:47:12 dist-test-slave-6ntk krb5kdc[20177](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988032, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:47:12.678776 12279 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 06 03:47:12 dist-test-slave-6ntk krb5kdc[20177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988032, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.253.193@KRBTEST.COM
I20260806 03:47:12.685099 20922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:12.679250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52328 (local address 127.11.253.193:45713)
0806 03:47:12.679307 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0806 03:47:12.679309 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:47:12.679324 (+ 15us) server_negotiation.cc:407] Connection header received
0806 03:47:12.679383 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:47:12.679383 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:47:12.679390 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:47:12.679427 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:47:12.679741 (+ 314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:12.680184 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:12.680824 (+ 640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:12.680995 (+ 171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:12.681041 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:12.683035 (+ 1994us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:47:12.683036 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:47:12.683036 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:47:12.683042 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:47:12.684167 (+ 1125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:47:12.684490 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:47:12.684491 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:47:12.684491 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:47:12.684537 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:47:12.684686 (+ 149us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:47:12.684687 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:47:12.684687 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:47:12.684765 (+ 78us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:47:12.684828 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0806 03:47:12.684915 (+ 87us) server_negotiation.cc:299] Negotiation successful
0806 03:47:12.684965 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260806 03:47:12.685756 12279 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskToVWRe/build/release/bin/kudu with pid 20341
I20260806 03:47:12.697791 12279 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskToVWRe/build/release/bin/kudu with pid 20535
I20260806 03:47:12.710583 12279 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskToVWRe/build/release/bin/kudu with pid 20729
I20260806 03:47:12.718537 12279 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskToVWRe/build/release/bin/kudu with pid 20196
2026-08-06T03:47:12Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (2149 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988004637995-12279-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 03:47:12 dist-test-slave-6ntk krb5kdc[20936](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 03:47:12 dist-test-slave-6ntk krb5kdc[20936](info): set up 2 sockets
Aug 06 03:47:12 dist-test-slave-6ntk krb5kdc[20936](info): commencing operation
krb5kdc: starting...
W20260806 03:47:13.949971 12279 mini_kdc.cc:121] Time spent starting KDC: real 1.204s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988004637995-12279-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988004637995-12279-0/krb5kdc/test-user.keytab.
Aug 06 03:47:14 dist-test-slave-6ntk krb5kdc[20936](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988034, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T03:47:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T03:47:14Z Disabled control of system clock
WARNING: no policy specified for oryx/127.11.253.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.11.253.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.11.253.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988004637995-12279-0/krb5kdc/oryx_127.11.253.254.keytab.
Entry for principal oryx/127.11.253.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988004637995-12279-0/krb5kdc/oryx_127.11.253.254.keytab.
I20260806 03:47:14.081142 12279 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskToVWRe/build/release/bin/kudu
/tmp/dist-test-taskToVWRe/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988004637995-12279-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988004637995-12279-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988004637995-12279-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988004637995-12279-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.253.254:32833
--webserver_interface=127.11.253.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988004637995-12279-0/krb5kdc/oryx_127.11.253.254.keytab
--principal=oryx/127.11.253.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.253.212:45893
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.253.254:32833
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988004637995-12279-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988004637995-12279-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988004637995-12279-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:47:14.167069 20957 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:47:14.167318 20957 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:47:14.167346 20957 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:47:14.168440 20957 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 03:47:14.168478 20957 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:47:14.168494 20957 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:47:14.168509 20957 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 03:47:14.168522 20957 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 03:47:14.169974 20957 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.253.212:45893
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988004637995-12279-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988004637995-12279-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.253.254:32833
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988004637995-12279-0/krb5kdc/oryx_127.11.253.254.keytab
--principal=oryx/127.11.253.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.253.254:32833
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988004637995-12279-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.253.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20957
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988004637995-12279-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:47:14.170185 20957 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:47:14.170378 20957 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:47:14.175673 20963 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:47:14.175644 20966 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:47:14.175652 20969 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:47:14.175684 20957 server_base.cc:1034] running on GCE node
I20260806 03:47:14.176120 20957 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:47:14.176380 20957 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:47:14.177533 20957 hybrid_clock.cc:648] HybridClock initialized: now 1785988034177507 us; error 45 us; skew 500 ppm
Aug 06 03:47:14 dist-test-slave-6ntk krb5kdc[20936](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988034, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.253.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:47:14.180202 20957 init.cc:377] Logged in from keytab as oryx/127.11.253.254@KRBTEST.COM (short username oryx)
I20260806 03:47:14.180680 20957 webserver.cc:460] Webserver started at http://127.11.253.254:40977/ using document root <none> and password file <none>
I20260806 03:47:14.180910 20957 fs_manager.cc:359] Metadata directory not provided
I20260806 03:47:14.180955 20957 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:47:14.181119 20957 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:47:14.182267 20957 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988004637995-12279-0/minicluster-data/master-0/data/instance:
uuid: "46d7cfc9ed574f1794039e138d8f6220"
format_stamp: "Formatted at 2026-08-06 03:47:14 on dist-test-slave-6ntk"
server_key: "3065235f3d1647d02e061ca0af4b1019"
server_key_iv: "0d7764be8cbe1040f7a949f80a4af3b9"
server_key_version: "encryptionkey@0"
I20260806 03:47:14.182636 20957 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988004637995-12279-0/minicluster-data/master-0/wal/instance:
uuid: "46d7cfc9ed574f1794039e138d8f6220"
format_stamp: "Formatted at 2026-08-06 03:47:14 on dist-test-slave-6ntk"
server_key: "3065235f3d1647d02e061ca0af4b1019"
server_key_iv: "0d7764be8cbe1040f7a949f80a4af3b9"
server_key_version: "encryptionkey@0"
I20260806 03:47:14.184068 20957 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 03:47:14.184793 20982 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:47:14.184990 20957 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 03:47:14.185093 20957 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988004637995-12279-0/minicluster-data/master-0/data,/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988004637995-12279-0/minicluster-data/master-0/wal
uuid: "46d7cfc9ed574f1794039e138d8f6220"
format_stamp: "Formatted at 2026-08-06 03:47:14 on dist-test-slave-6ntk"
server_key: "3065235f3d1647d02e061ca0af4b1019"
server_key_iv: "0d7764be8cbe1040f7a949f80a4af3b9"
server_key_version: "encryptionkey@0"
I20260806 03:47:14.185169 20957 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988004637995-12279-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988004637995-12279-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988004637995-12279-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:47:14.234117 20957 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:47:14.236265 20957 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:47:14.236428 20957 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:47:14.240911 20957 rpc_server.cc:307] RPC server started. Bound to: 127.11.253.254:32833
I20260806 03:47:14.240963 21054 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.253.254:32833 every 8 connection(s)
I20260806 03:47:14.241338 20957 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988004637995-12279-0/minicluster-data/master-0/data/info.pb
I20260806 03:47:14.242527 21055 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:47:14.245659 21055 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 46d7cfc9ed574f1794039e138d8f6220: Bootstrap starting.
I20260806 03:47:14.246326 21055 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 46d7cfc9ed574f1794039e138d8f6220: Neither blocks nor log segments found. Creating new log.
I20260806 03:47:14.246603 21055 log.cc:824] T 00000000000000000000000000000000 P 46d7cfc9ed574f1794039e138d8f6220: Log is configured to *not* fsync() on all Append() calls
I20260806 03:47:14.247411 21055 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 46d7cfc9ed574f1794039e138d8f6220: No bootstrap required, opened a new log
I20260806 03:47:14.247412 12279 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskToVWRe/build/release/bin/kudu as pid 20957
I20260806 03:47:14.247520 12279 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988004637995-12279-0/minicluster-data/master-0/wal/instance
I20260806 03:47:14.247772 12279 external_mini_cluster.cc:1428] Setting key 1a4f0975173c6dfa042c368a85613a33
I20260806 03:47:14.249482 21055 raft_consensus.cc:348] T 00000000000000000000000000000000 P 46d7cfc9ed574f1794039e138d8f6220 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46d7cfc9ed574f1794039e138d8f6220" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 32833 } }
I20260806 03:47:14.249625 21055 raft_consensus.cc:374] T 00000000000000000000000000000000 P 46d7cfc9ed574f1794039e138d8f6220 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:47:14.249650 21055 raft_consensus.cc:729] T 00000000000000000000000000000000 P 46d7cfc9ed574f1794039e138d8f6220 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 46d7cfc9ed574f1794039e138d8f6220, State: Initialized, Role: FOLLOWER
I20260806 03:47:14.249887 21055 consensus_queue.cc:260] T 00000000000000000000000000000000 P 46d7cfc9ed574f1794039e138d8f6220 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46d7cfc9ed574f1794039e138d8f6220" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 32833 } }
I20260806 03:47:14.249993 21055 raft_consensus.cc:388] T 00000000000000000000000000000000 P 46d7cfc9ed574f1794039e138d8f6220 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 03:47:14.250031 21055 raft_consensus.cc:482] T 00000000000000000000000000000000 P 46d7cfc9ed574f1794039e138d8f6220 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 03:47:14.250097 21055 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 46d7cfc9ed574f1794039e138d8f6220 [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:47:14.251272 21055 raft_consensus.cc:504] T 00000000000000000000000000000000 P 46d7cfc9ed574f1794039e138d8f6220 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46d7cfc9ed574f1794039e138d8f6220" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 32833 } }
I20260806 03:47:14.251374 21055 leader_election.cc:302] T 00000000000000000000000000000000 P 46d7cfc9ed574f1794039e138d8f6220 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 46d7cfc9ed574f1794039e138d8f6220; no voters:
I20260806 03:47:14.251561 21055 leader_election.cc:288] T 00000000000000000000000000000000 P 46d7cfc9ed574f1794039e138d8f6220 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 03:47:14.251603 21062 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 46d7cfc9ed574f1794039e138d8f6220 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 03:47:14.251768 21062 raft_consensus.cc:686] T 00000000000000000000000000000000 P 46d7cfc9ed574f1794039e138d8f6220 [term 1 LEADER]: Becoming Leader. State: Replica: 46d7cfc9ed574f1794039e138d8f6220, State: Running, Role: LEADER
I20260806 03:47:14.251837 21055 sys_catalog.cc:564] T 00000000000000000000000000000000 P 46d7cfc9ed574f1794039e138d8f6220 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 03:47:14.252005 21062 consensus_queue.cc:237] T 00000000000000000000000000000000 P 46d7cfc9ed574f1794039e138d8f6220 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46d7cfc9ed574f1794039e138d8f6220" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 32833 } }
I20260806 03:47:14.253218 21065 sys_catalog.cc:455] T 00000000000000000000000000000000 P 46d7cfc9ed574f1794039e138d8f6220 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 46d7cfc9ed574f1794039e138d8f6220. Latest consensus state: current_term: 1 leader_uuid: "46d7cfc9ed574f1794039e138d8f6220" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46d7cfc9ed574f1794039e138d8f6220" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 32833 } } }
I20260806 03:47:14.253314 21065 sys_catalog.cc:458] T 00000000000000000000000000000000 P 46d7cfc9ed574f1794039e138d8f6220 [sys.catalog]: This master's current role is: LEADER
I20260806 03:47:14.253698 21063 sys_catalog.cc:455] T 00000000000000000000000000000000 P 46d7cfc9ed574f1794039e138d8f6220 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "46d7cfc9ed574f1794039e138d8f6220" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46d7cfc9ed574f1794039e138d8f6220" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 32833 } } }
I20260806 03:47:14.253906 21063 sys_catalog.cc:458] T 00000000000000000000000000000000 P 46d7cfc9ed574f1794039e138d8f6220 [sys.catalog]: This master's current role is: LEADER
W20260806 03:47:14.253939 21088 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 46d7cfc9ed574f1794039e138d8f6220: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 03:47:14.254078 21088 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 03:47:14.254493 21090 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 03:47:14.254734 21090 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 06 03:47:14 dist-test-slave-6ntk krb5kdc[20936](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988034, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.11.253.254@KRBTEST.COM
I20260806 03:47:14.256975 21090 catalog_manager.cc:1347] Generated new cluster ID: 21e5bafe25394fe580e3c98318fe43c7
I20260806 03:47:14.257043 21090 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 03:47:14.261354 21061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:14.248322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42146 (local address 127.11.253.254:32833)
0806 03:47:14.248509 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0806 03:47:14.248517 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 03:47:14.248553 (+ 36us) server_negotiation.cc:407] Connection header received
0806 03:47:14.249487 (+ 934us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:47:14.249495 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:47:14.249549 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:47:14.249746 (+ 197us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:47:14.250666 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:14.251783 (+ 1117us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:14.252918 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:14.253639 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:14.254137 (+ 498us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:14.256933 (+ 2796us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:47:14.256937 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:47:14.256938 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:47:14.256951 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:47:14.259588 (+ 2637us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:47:14.260045 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:47:14.260049 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:47:14.260049 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:47:14.260183 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:47:14.260430 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:47:14.260431 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:47:14.260431 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:47:14.260731 (+ 300us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:47:14.260894 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0806 03:47:14.261003 (+ 109us) server_negotiation.cc:299] Negotiation successful
0806 03:47:14.261137 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":108,"threads_started":1}
I20260806 03:47:14.287708 21090 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 03:47:14.288291 21090 catalog_manager.cc:1504] Loading token signing keys...
I20260806 03:47:14.292271 21090 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 46d7cfc9ed574f1794039e138d8f6220: Generated new TSK 0
I20260806 03:47:14.292450 21090 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.11.253.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.11.253.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.11.253.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988004637995-12279-0/krb5kdc/oryx_127.11.253.193.keytab.
Entry for principal oryx/127.11.253.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988004637995-12279-0/krb5kdc/oryx_127.11.253.193.keytab.
I20260806 03:47:14.329164 12279 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskToVWRe/build/release/bin/kudu
/tmp/dist-test-taskToVWRe/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988004637995-12279-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988004637995-12279-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988004637995-12279-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988004637995-12279-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.253.193:0
--local_ip_for_outbound_sockets=127.11.253.193
--webserver_interface=127.11.253.193
--webserver_port=0
--tserver_master_addrs=127.11.253.254:32833
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988004637995-12279-0/krb5kdc/oryx_127.11.253.193.keytab
--principal=oryx/127.11.253.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.253.212:45893
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988004637995-12279-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988004637995-12279-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988004637995-12279-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:47:14.414132 21096 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:47:14.414376 21096 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:47:14.414415 21096 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:47:14.415587 21096 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:47:14.415660 21096 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:47:14.415707 21096 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.253.193
I20260806 03:47:14.417229 21096 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.253.212:45893
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988004637995-12279-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988004637995-12279-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988004637995-12279-0/krb5kdc/oryx_127.11.253.193.keytab
--principal=oryx/127.11.253.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.253.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988004637995-12279-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.253.193
--webserver_port=0
--tserver_master_addrs=127.11.253.254:32833
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21096
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.253.193
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988004637995-12279-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:47:14.417496 21096 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:47:14.417737 21096 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:47:14.423010 21106 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:47:14.423013 21103 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:47:14.423203 21104 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:47:14.423352 21096 server_base.cc:1034] running on GCE node
I20260806 03:47:14.423569 21096 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:47:14.423835 21096 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:47:14.424983 21096 hybrid_clock.cc:648] HybridClock initialized: now 1785988034424964 us; error 36 us; skew 500 ppm
Aug 06 03:47:14 dist-test-slave-6ntk krb5kdc[20936](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988034, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.253.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:47:14.427614 21096 init.cc:377] Logged in from keytab as oryx/127.11.253.193@KRBTEST.COM (short username oryx)
I20260806 03:47:14.428076 21096 webserver.cc:460] Webserver started at http://127.11.253.193:35453/ using document root <none> and password file <none>
I20260806 03:47:14.428262 21096 fs_manager.cc:359] Metadata directory not provided
I20260806 03:47:14.428304 21096 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:47:14.428393 21096 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:47:14.429389 21096 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988004637995-12279-0/minicluster-data/ts-0/data/instance:
uuid: "21d27ac5b0aa4681b44bc0482b668d54"
format_stamp: "Formatted at 2026-08-06 03:47:14 on dist-test-slave-6ntk"
server_key: "4ca1b394c8127be478d17a105e99a79b"
server_key_iv: "2b4a03b4ec313f9ba6c43b4f53089da7"
server_key_version: "encryptionkey@0"
I20260806 03:47:14.429692 21096 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988004637995-12279-0/minicluster-data/ts-0/wal/instance:
uuid: "21d27ac5b0aa4681b44bc0482b668d54"
format_stamp: "Formatted at 2026-08-06 03:47:14 on dist-test-slave-6ntk"
server_key: "4ca1b394c8127be478d17a105e99a79b"
server_key_iv: "2b4a03b4ec313f9ba6c43b4f53089da7"
server_key_version: "encryptionkey@0"
I20260806 03:47:14.431041 21096 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 03:47:14.431728 21114 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:47:14.431916 21096 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 03:47:14.432020 21096 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988004637995-12279-0/minicluster-data/ts-0/data,/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988004637995-12279-0/minicluster-data/ts-0/wal
uuid: "21d27ac5b0aa4681b44bc0482b668d54"
format_stamp: "Formatted at 2026-08-06 03:47:14 on dist-test-slave-6ntk"
server_key: "4ca1b394c8127be478d17a105e99a79b"
server_key_iv: "2b4a03b4ec313f9ba6c43b4f53089da7"
server_key_version: "encryptionkey@0"
I20260806 03:47:14.432089 21096 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988004637995-12279-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988004637995-12279-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988004637995-12279-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:47:14.444774 21096 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:47:14.447132 21096 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:47:14.447299 21096 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:47:14.447535 21096 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 03:47:14.447866 21096 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 03:47:14.447921 21096 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 03:47:14.447957 21096 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 03:47:14.447983 21096 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 03:47:14.454027 21096 rpc_server.cc:307] RPC server started. Bound to: 127.11.253.193:34025
I20260806 03:47:14.454080 21277 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.253.193:34025 every 8 connection(s)
I20260806 03:47:14.454394 21096 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988004637995-12279-0/minicluster-data/ts-0/data/info.pb
I20260806 03:47:14.455245 12279 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskToVWRe/build/release/bin/kudu as pid 21096
I20260806 03:47:14.455367 12279 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988004637995-12279-0/minicluster-data/ts-0/wal/instance
I20260806 03:47:14.455562 12279 external_mini_cluster.cc:1428] Setting key 668b99bee23851ce52fb503a74b38db1
Aug 06 03:47:14 dist-test-slave-6ntk krb5kdc[20936](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988034, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.253.193@KRBTEST.COM for oryx/127.11.253.254@KRBTEST.COM
WARNING: no policy specified for oryx/127.11.253.194@KRBTEST.COM; defaulting to no policy
I20260806 03:47:14.464998 21061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:14.455096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.253.193:34615 (local address 127.11.253.254:32833)
0806 03:47:14.455144 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0806 03:47:14.455147 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 03:47:14.455521 (+ 374us) server_negotiation.cc:407] Connection header received
0806 03:47:14.456162 (+ 641us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:47:14.456163 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:47:14.456176 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:47:14.456222 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:47:14.457175 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:14.457675 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:14.459254 (+ 1579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:14.459471 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:14.459536 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:14.462052 (+ 2516us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:47:14.462054 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:47:14.462055 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:47:14.462064 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:47:14.463533 (+ 1469us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:47:14.463965 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:47:14.463966 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:47:14.463966 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:47:14.464024 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:47:14.464274 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:47:14.464275 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:47:14.464275 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:47:14.464393 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:47:14.464463 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0806 03:47:14.464760 (+ 297us) server_negotiation.cc:299] Negotiation successful
0806 03:47:14.464861 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260806 03:47:14.465143 21282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:14.455097 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:32833 (local address 127.11.253.193:34615)
0806 03:47:14.455375 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0806 03:47:14.455426 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0806 03:47:14.456071 (+ 645us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:47:14.456300 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:47:14.456304 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:47:14.456884 (+ 580us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:47:14.457073 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:14.457077 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:14.457817 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:14.457819 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:14.459156 (+ 1337us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:14.459159 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:14.459565 (+ 406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:14.459566 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:14.459715 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:14.460269 (+ 554us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:47:14.460280 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:47:14.461962 (+ 1682us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:47:14.463642 (+ 1680us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:47:14.463649 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:47:14.463651 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:47:14.463882 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:47:14.464093 (+ 211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:47:14.464094 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:47:14.464094 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:47:14.464204 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:47:14.464464 (+ 260us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:47:14.464472 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:47:14.464610 (+ 138us) client_negotiation.cc:769] Sending connection context
0806 03:47:14.464718 (+ 108us) client_negotiation.cc:240] Negotiation successful
0806 03:47:14.464884 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":231,"thread_start_us":182,"threads_started":1}
I20260806 03:47:14.465641 21279 heartbeater.cc:347] Connected to a master server at 127.11.253.254:32833
I20260806 03:47:14.465785 21279 heartbeater.cc:464] Registering TS with master...
I20260806 03:47:14.466049 21279 heartbeater.cc:511] Master 127.11.253.254:32833 requested a full tablet report, sending...
I20260806 03:47:14.466545 21005 ts_manager.cc:194] Registered new tserver with Master: 21d27ac5b0aa4681b44bc0482b668d54 (127.11.253.193:34025)
I20260806 03:47:14.467216 21005 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.11.253.193@KRBTEST.COM'} at 127.11.253.193:34615
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.11.253.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.11.253.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988004637995-12279-0/krb5kdc/oryx_127.11.253.194.keytab.
Entry for principal oryx/127.11.253.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988004637995-12279-0/krb5kdc/oryx_127.11.253.194.keytab.
I20260806 03:47:14.484186 12279 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskToVWRe/build/release/bin/kudu
/tmp/dist-test-taskToVWRe/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988004637995-12279-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988004637995-12279-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988004637995-12279-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988004637995-12279-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.253.194:0
--local_ip_for_outbound_sockets=127.11.253.194
--webserver_interface=127.11.253.194
--webserver_port=0
--tserver_master_addrs=127.11.253.254:32833
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988004637995-12279-0/krb5kdc/oryx_127.11.253.194.keytab
--principal=oryx/127.11.253.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.253.212:45893
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988004637995-12279-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988004637995-12279-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988004637995-12279-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:47:14.567216 21285 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:47:14.567433 21285 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:47:14.567458 21285 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:47:14.568589 21285 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:47:14.568630 21285 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:47:14.568676 21285 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.253.194
I20260806 03:47:14.570127 21285 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.253.212:45893
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988004637995-12279-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988004637995-12279-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988004637995-12279-0/krb5kdc/oryx_127.11.253.194.keytab
--principal=oryx/127.11.253.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.253.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988004637995-12279-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.253.194
--webserver_port=0
--tserver_master_addrs=127.11.253.254:32833
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21285
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.253.194
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988004637995-12279-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:47:14.570369 21285 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:47:14.570603 21285 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:47:14.575697 21293 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:47:14.575706 21294 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:47:14.575853 21285 server_base.cc:1034] running on GCE node
W20260806 03:47:14.575771 21298 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:47:14.576133 21285 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:47:14.576387 21285 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:47:14.577536 21285 hybrid_clock.cc:648] HybridClock initialized: now 1785988034577518 us; error 35 us; skew 500 ppm
Aug 06 03:47:14 dist-test-slave-6ntk krb5kdc[20936](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988034, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.253.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:47:14.580084 21285 init.cc:377] Logged in from keytab as oryx/127.11.253.194@KRBTEST.COM (short username oryx)
I20260806 03:47:14.580616 21285 webserver.cc:460] Webserver started at http://127.11.253.194:46449/ using document root <none> and password file <none>
I20260806 03:47:14.580842 21285 fs_manager.cc:359] Metadata directory not provided
I20260806 03:47:14.580906 21285 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:47:14.581053 21285 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:47:14.582213 21285 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988004637995-12279-0/minicluster-data/ts-1/data/instance:
uuid: "0fa8b770e150486aae31b4df6ad38a6e"
format_stamp: "Formatted at 2026-08-06 03:47:14 on dist-test-slave-6ntk"
server_key: "cbc31c6e3f3148dd1569cd5d80a5e891"
server_key_iv: "d07befe1b47a3bd7b2c79c2ba7fd8abb"
server_key_version: "encryptionkey@0"
I20260806 03:47:14.582576 21285 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988004637995-12279-0/minicluster-data/ts-1/wal/instance:
uuid: "0fa8b770e150486aae31b4df6ad38a6e"
format_stamp: "Formatted at 2026-08-06 03:47:14 on dist-test-slave-6ntk"
server_key: "cbc31c6e3f3148dd1569cd5d80a5e891"
server_key_iv: "d07befe1b47a3bd7b2c79c2ba7fd8abb"
server_key_version: "encryptionkey@0"
I20260806 03:47:14.583892 21285 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 03:47:14.584591 21310 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:47:14.584750 21285 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 03:47:14.584870 21285 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988004637995-12279-0/minicluster-data/ts-1/data,/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988004637995-12279-0/minicluster-data/ts-1/wal
uuid: "0fa8b770e150486aae31b4df6ad38a6e"
format_stamp: "Formatted at 2026-08-06 03:47:14 on dist-test-slave-6ntk"
server_key: "cbc31c6e3f3148dd1569cd5d80a5e891"
server_key_iv: "d07befe1b47a3bd7b2c79c2ba7fd8abb"
server_key_version: "encryptionkey@0"
I20260806 03:47:14.584942 21285 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988004637995-12279-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988004637995-12279-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988004637995-12279-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:47:14.601341 21285 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:47:14.603955 21285 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:47:14.604108 21285 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:47:14.604351 21285 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 03:47:14.604671 21285 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 03:47:14.604725 21285 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 03:47:14.604768 21285 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 03:47:14.604795 21285 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 03:47:14.610862 21285 rpc_server.cc:307] RPC server started. Bound to: 127.11.253.194:45307
I20260806 03:47:14.610900 21462 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.253.194:45307 every 8 connection(s)
I20260806 03:47:14.611238 21285 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988004637995-12279-0/minicluster-data/ts-1/data/info.pb
Aug 06 03:47:14 dist-test-slave-6ntk krb5kdc[20936](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988034, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.253.194@KRBTEST.COM for oryx/127.11.253.254@KRBTEST.COM
I20260806 03:47:14.619632 12279 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskToVWRe/build/release/bin/kudu as pid 21285
I20260806 03:47:14.619719 12279 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988004637995-12279-0/minicluster-data/ts-1/wal/instance
I20260806 03:47:14.619918 12279 external_mini_cluster.cc:1428] Setting key e1e93644151b62f73f43e777aa8fc2bb
I20260806 03:47:14.620919 21061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:14.612011 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.253.194:51761 (local address 127.11.253.254:32833)
0806 03:47:14.612064 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0806 03:47:14.612066 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:47:14.612250 (+ 184us) server_negotiation.cc:407] Connection header received
0806 03:47:14.612724 (+ 474us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:47:14.612725 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:47:14.612734 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:47:14.612792 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:47:14.613744 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:14.614298 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:14.615245 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:14.615444 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:14.615495 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:14.617601 (+ 2106us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:47:14.617603 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:47:14.617603 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:47:14.617610 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:47:14.619041 (+ 1431us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:47:14.619573 (+ 532us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:47:14.619575 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:47:14.619575 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:47:14.619744 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:47:14.620094 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:47:14.620096 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:47:14.620096 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:47:14.620259 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:47:14.620336 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0806 03:47:14.620696 (+ 360us) server_negotiation.cc:299] Negotiation successful
0806 03:47:14.620790 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260806 03:47:14.621065 21465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:14.611958 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:32833 (local address 127.11.253.194:51761)
0806 03:47:14.612124 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0806 03:47:14.612166 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 03:47:14.612646 (+ 480us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:47:14.612896 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:47:14.612900 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:47:14.613476 (+ 576us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:47:14.613665 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:14.613670 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:14.614379 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:14.614380 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:14.615168 (+ 788us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:14.615170 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:14.615513 (+ 343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:14.615514 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:14.615610 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:14.616075 (+ 465us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:47:14.616083 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:47:14.617487 (+ 1404us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:47:14.619143 (+ 1656us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:47:14.619150 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:47:14.619153 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:47:14.619480 (+ 327us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:47:14.619865 (+ 385us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:47:14.619866 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:47:14.619867 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:47:14.620013 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:47:14.620360 (+ 347us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:47:14.620368 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:47:14.620540 (+ 172us) client_negotiation.cc:769] Sending connection context
0806 03:47:14.620668 (+ 128us) client_negotiation.cc:240] Negotiation successful
0806 03:47:14.620855 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":89,"threads_started":1}
I20260806 03:47:14.621517 21463 heartbeater.cc:347] Connected to a master server at 127.11.253.254:32833
I20260806 03:47:14.621631 21463 heartbeater.cc:464] Registering TS with master...
I20260806 03:47:14.621831 21463 heartbeater.cc:511] Master 127.11.253.254:32833 requested a full tablet report, sending...
I20260806 03:47:14.622301 21005 ts_manager.cc:194] Registered new tserver with Master: 0fa8b770e150486aae31b4df6ad38a6e (127.11.253.194:45307)
I20260806 03:47:14.622824 21005 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.11.253.194@KRBTEST.COM'} at 127.11.253.194:51761
WARNING: no policy specified for oryx/127.11.253.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.11.253.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.11.253.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988004637995-12279-0/krb5kdc/oryx_127.11.253.195.keytab.
Entry for principal oryx/127.11.253.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988004637995-12279-0/krb5kdc/oryx_127.11.253.195.keytab.
I20260806 03:47:14.649927 12279 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskToVWRe/build/release/bin/kudu
/tmp/dist-test-taskToVWRe/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988004637995-12279-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988004637995-12279-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988004637995-12279-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988004637995-12279-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.253.195:0
--local_ip_for_outbound_sockets=127.11.253.195
--webserver_interface=127.11.253.195
--webserver_port=0
--tserver_master_addrs=127.11.253.254:32833
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988004637995-12279-0/krb5kdc/oryx_127.11.253.195.keytab
--principal=oryx/127.11.253.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.253.212:45893
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988004637995-12279-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988004637995-12279-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988004637995-12279-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:47:14.734216 21472 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:47:14.734433 21472 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:47:14.734465 21472 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:47:14.735736 21472 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:47:14.735782 21472 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:47:14.735808 21472 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.253.195
I20260806 03:47:14.737262 21472 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.253.212:45893
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988004637995-12279-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988004637995-12279-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988004637995-12279-0/krb5kdc/oryx_127.11.253.195.keytab
--principal=oryx/127.11.253.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.253.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988004637995-12279-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.253.195
--webserver_port=0
--tserver_master_addrs=127.11.253.254:32833
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21472
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.253.195
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988004637995-12279-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:47:14.737525 21472 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:47:14.737779 21472 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:47:14.742920 21478 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:47:14.742969 21477 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:47:14.742978 21482 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:47:14.743257 21472 server_base.cc:1034] running on GCE node
I20260806 03:47:14.743660 21472 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:47:14.743932 21472 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:47:14.745124 21472 hybrid_clock.cc:648] HybridClock initialized: now 1785988034745108 us; error 41 us; skew 500 ppm
Aug 06 03:47:14 dist-test-slave-6ntk krb5kdc[20936](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988034, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.253.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:47:14.747856 21472 init.cc:377] Logged in from keytab as oryx/127.11.253.195@KRBTEST.COM (short username oryx)
I20260806 03:47:14.748355 21472 webserver.cc:460] Webserver started at http://127.11.253.195:34977/ using document root <none> and password file <none>
I20260806 03:47:14.748607 21472 fs_manager.cc:359] Metadata directory not provided
I20260806 03:47:14.748674 21472 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:47:14.748822 21472 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:47:14.749908 21472 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988004637995-12279-0/minicluster-data/ts-2/data/instance:
uuid: "05f018b6c7cd4ee3a66404e6d7ef0729"
format_stamp: "Formatted at 2026-08-06 03:47:14 on dist-test-slave-6ntk"
server_key: "5aeefb93b1f28cd79d07baef53cf30dd"
server_key_iv: "5cbf1f033813c7a21e98a05e8bc060b2"
server_key_version: "encryptionkey@0"
I20260806 03:47:14.750283 21472 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988004637995-12279-0/minicluster-data/ts-2/wal/instance:
uuid: "05f018b6c7cd4ee3a66404e6d7ef0729"
format_stamp: "Formatted at 2026-08-06 03:47:14 on dist-test-slave-6ntk"
server_key: "5aeefb93b1f28cd79d07baef53cf30dd"
server_key_iv: "5cbf1f033813c7a21e98a05e8bc060b2"
server_key_version: "encryptionkey@0"
I20260806 03:47:14.751685 21472 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 03:47:14.752487 21493 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:47:14.752650 21472 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 03:47:14.752755 21472 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988004637995-12279-0/minicluster-data/ts-2/data,/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988004637995-12279-0/minicluster-data/ts-2/wal
uuid: "05f018b6c7cd4ee3a66404e6d7ef0729"
format_stamp: "Formatted at 2026-08-06 03:47:14 on dist-test-slave-6ntk"
server_key: "5aeefb93b1f28cd79d07baef53cf30dd"
server_key_iv: "5cbf1f033813c7a21e98a05e8bc060b2"
server_key_version: "encryptionkey@0"
I20260806 03:47:14.752839 21472 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988004637995-12279-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988004637995-12279-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988004637995-12279-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:47:14.763084 21472 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:47:14.764933 21472 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:47:14.765089 21472 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:47:14.765313 21472 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 03:47:14.765668 21472 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 03:47:14.765718 21472 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 03:47:14.765760 21472 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 03:47:14.765787 21472 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 03:47:14.771565 21472 rpc_server.cc:307] RPC server started. Bound to: 127.11.253.195:33553
I20260806 03:47:14.771633 21659 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.253.195:33553 every 8 connection(s)
I20260806 03:47:14.771956 21472 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988004637995-12279-0/minicluster-data/ts-2/data/info.pb
I20260806 03:47:14.775655 12279 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskToVWRe/build/release/bin/kudu as pid 21472
I20260806 03:47:14.775753 12279 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988004637995-12279-0/minicluster-data/ts-2/wal/instance
I20260806 03:47:14.775918 12279 external_mini_cluster.cc:1428] Setting key 70c4d1b99bd8a6fdb72d90c579e51af7
Aug 06 03:47:14 dist-test-slave-6ntk krb5kdc[20936](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988034, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.253.195@KRBTEST.COM for oryx/127.11.253.254@KRBTEST.COM
I20260806 03:47:14.781370 21061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:14.772714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.253.195:56615 (local address 127.11.253.254:32833)
0806 03:47:14.772775 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0806 03:47:14.772777 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:47:14.772992 (+ 215us) server_negotiation.cc:407] Connection header received
0806 03:47:14.773527 (+ 535us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:47:14.773529 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:47:14.773543 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:47:14.773594 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:47:14.774610 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:14.775125 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:14.776241 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:14.776449 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:14.776521 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:14.778764 (+ 2243us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:47:14.778766 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:47:14.778766 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:47:14.778774 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:47:14.779987 (+ 1213us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:47:14.780401 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:47:14.780403 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:47:14.780403 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:47:14.780467 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:47:14.780702 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:47:14.780703 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:47:14.780703 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:47:14.780823 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:47:14.780915 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0806 03:47:14.781194 (+ 279us) server_negotiation.cc:299] Negotiation successful
0806 03:47:14.781278 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260806 03:47:14.781558 21662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:14.772691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:32833 (local address 127.11.253.195:56615)
0806 03:47:14.772878 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0806 03:47:14.772914 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0806 03:47:14.773417 (+ 503us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:47:14.773667 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:47:14.773671 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:47:14.774318 (+ 647us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:47:14.774513 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:14.774518 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:14.775223 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:14.775224 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:14.776146 (+ 922us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:14.776149 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:14.776528 (+ 379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:14.776529 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:14.776662 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:14.777172 (+ 510us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:47:14.777180 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:47:14.778656 (+ 1476us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:47:14.780075 (+ 1419us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:47:14.780082 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:47:14.780084 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:47:14.780304 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:47:14.780531 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:47:14.780532 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:47:14.780532 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:47:14.780636 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:47:14.780918 (+ 282us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:47:14.780926 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:47:14.781061 (+ 135us) client_negotiation.cc:769] Sending connection context
0806 03:47:14.781160 (+ 99us) client_negotiation.cc:240] Negotiation successful
0806 03:47:14.781320 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":114,"threads_started":1}
I20260806 03:47:14.782035 21660 heartbeater.cc:347] Connected to a master server at 127.11.253.254:32833
I20260806 03:47:14.782145 21660 heartbeater.cc:464] Registering TS with master...
I20260806 03:47:14.782297 21660 heartbeater.cc:511] Master 127.11.253.254:32833 requested a full tablet report, sending...
I20260806 03:47:14.782665 21005 ts_manager.cc:194] Registered new tserver with Master: 05f018b6c7cd4ee3a66404e6d7ef0729 (127.11.253.195:33553)
I20260806 03:47:14.783066 21005 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.11.253.195@KRBTEST.COM'} at 127.11.253.195:56615
I20260806 03:47:14.787552 12279 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.11.253.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.11.253.253@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.11.253.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988004637995-12279-0/krb5kdc/oryx_127.11.253.253.keytab.
Entry for principal oryx/127.11.253.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988004637995-12279-0/krb5kdc/oryx_127.11.253.253.keytab.
I20260806 03:47:14.816190 12279 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskToVWRe/build/release/bin/kudu
/tmp/dist-test-taskToVWRe/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988004637995-12279-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988004637995-12279-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988004637995-12279-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988004637995-12279-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.253.253:46317
--webserver_interface=127.11.253.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988004637995-12279-0/krb5kdc/oryx_127.11.253.253.keytab
--principal=oryx/127.11.253.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.253.212:45893
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.253.254:32833,127.11.253.253:46317
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988004637995-12279-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988004637995-12279-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988004637995-12279-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:47:14.900785 21667 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:47:14.901044 21667 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:47:14.901081 21667 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:47:14.902307 21667 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 03:47:14.902367 21667 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:47:14.902390 21667 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:47:14.902442 21667 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 03:47:14.902473 21667 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 03:47:14.903944 21667 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.253.212:45893
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988004637995-12279-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988004637995-12279-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.11.253.254:32833,127.11.253.253:46317
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988004637995-12279-0/krb5kdc/oryx_127.11.253.253.keytab
--principal=oryx/127.11.253.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.253.253:46317
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988004637995-12279-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.11.253.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21667
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988004637995-12279-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
W20260806 03:47:14.904088 21667 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.11.253.254:32833,127.11.253.253:46317'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260806 03:47:14.904251 21667 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:47:14.904475 21667 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:47:14.909309 21674 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:47:14.909292 21675 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:47:14.909317 21682 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:47:14.909653 21667 server_base.cc:1034] running on GCE node
I20260806 03:47:14.909984 21667 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:47:14.910264 21667 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:47:14.911415 21667 hybrid_clock.cc:648] HybridClock initialized: now 1785988034911402 us; error 34 us; skew 500 ppm
Aug 06 03:47:14 dist-test-slave-6ntk krb5kdc[20936](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988034, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.253.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:47:14.913910 21667 init.cc:377] Logged in from keytab as oryx/127.11.253.253@KRBTEST.COM (short username oryx)
I20260806 03:47:14.914332 21667 webserver.cc:460] Webserver started at http://127.11.253.253:41629/ using document root <none> and password file <none>
I20260806 03:47:14.914552 21667 fs_manager.cc:359] Metadata directory not provided
I20260806 03:47:14.914618 21667 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:47:14.914765 21667 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:47:14.915797 21667 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988004637995-12279-0/minicluster-data/master-1/data/instance:
uuid: "85510334aee348ffbc06964d94b3087b"
format_stamp: "Formatted at 2026-08-06 03:47:14 on dist-test-slave-6ntk"
server_key: "0bfe95f85bde7fc653f7d315853c665c"
server_key_iv: "e20d73f2d23538d44c46cf0363283d38"
server_key_version: "encryptionkey@0"
I20260806 03:47:14.916172 21667 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988004637995-12279-0/minicluster-data/master-1/wal/instance:
uuid: "85510334aee348ffbc06964d94b3087b"
format_stamp: "Formatted at 2026-08-06 03:47:14 on dist-test-slave-6ntk"
server_key: "0bfe95f85bde7fc653f7d315853c665c"
server_key_iv: "e20d73f2d23538d44c46cf0363283d38"
server_key_version: "encryptionkey@0"
I20260806 03:47:14.917513 21667 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260806 03:47:14.918290 21692 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:47:14.918452 21667 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 03:47:14.918550 21667 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988004637995-12279-0/minicluster-data/master-1/data,/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988004637995-12279-0/minicluster-data/master-1/wal
uuid: "85510334aee348ffbc06964d94b3087b"
format_stamp: "Formatted at 2026-08-06 03:47:14 on dist-test-slave-6ntk"
server_key: "0bfe95f85bde7fc653f7d315853c665c"
server_key_iv: "e20d73f2d23538d44c46cf0363283d38"
server_key_version: "encryptionkey@0"
I20260806 03:47:14.918625 21667 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988004637995-12279-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988004637995-12279-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988004637995-12279-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:47:14.940018 21667 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:47:14.941970 21667 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:47:14.942134 21667 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:47:14.946084 21667 rpc_server.cc:307] RPC server started. Bound to: 127.11.253.253:46317
I20260806 03:47:14.946117 21764 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.253.253:46317 every 8 connection(s)
I20260806 03:47:14.946492 21667 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988004637995-12279-0/minicluster-data/master-1/data/info.pb
I20260806 03:47:14.947535 21765 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:47:14.949018 21765 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.253.254" port: 32833 } has no permanent_uuid. Determining permanent_uuid...
I20260806 03:47:14.951573 12279 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskToVWRe/build/release/bin/kudu as pid 21667
Aug 06 03:47:14 dist-test-slave-6ntk krb5kdc[20936](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988034, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.253.253@KRBTEST.COM for oryx/127.11.253.254@KRBTEST.COM
Aug 06 03:47:14 dist-test-slave-6ntk krb5kdc[20936](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988034, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.11.253.253@KRBTEST.COM
I20260806 03:47:14.958738 21061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:14.949705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42154 (local address 127.11.253.254:32833)
0806 03:47:14.949770 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0806 03:47:14.949772 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:47:14.950034 (+ 262us) server_negotiation.cc:407] Connection header received
0806 03:47:14.950563 (+ 529us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:47:14.950565 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:47:14.950577 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:47:14.950629 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:47:14.951241 (+ 612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:14.951772 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:14.953014 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:14.953238 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:14.953315 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:14.955864 (+ 2549us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:47:14.955866 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:47:14.955866 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:47:14.955873 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:47:14.957322 (+ 1449us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:47:14.957717 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:47:14.957718 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:47:14.957719 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:47:14.957785 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:47:14.958053 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:47:14.958054 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:47:14.958054 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:47:14.958208 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:47:14.958281 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0806 03:47:14.958561 (+ 280us) server_negotiation.cc:299] Negotiation successful
0806 03:47:14.958648 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260806 03:47:14.958906 21773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:14.949677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:32833 (local address 127.0.0.1:42154)
0806 03:47:14.949901 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0806 03:47:14.949952 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0806 03:47:14.950447 (+ 495us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:47:14.950700 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:47:14.950704 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:47:14.950953 (+ 249us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:47:14.951148 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:14.951152 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:14.951863 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:14.951864 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:14.952925 (+ 1061us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:14.952928 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:14.953321 (+ 393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:14.953323 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:14.953456 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:14.954042 (+ 586us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:47:14.954050 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:47:14.955777 (+ 1727us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:47:14.957401 (+ 1624us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:47:14.957408 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:47:14.957411 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:47:14.957632 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:47:14.957852 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:47:14.957853 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:47:14.957854 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:47:14.957985 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:47:14.958282 (+ 297us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:47:14.958289 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:47:14.958429 (+ 140us) client_negotiation.cc:769] Sending connection context
0806 03:47:14.958526 (+ 97us) client_negotiation.cc:240] Negotiation successful
0806 03:47:14.958673 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"mutex_wait_us":3,"thread_start_us":102,"threads_started":1}
I20260806 03:47:14.959551 21765 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.253.253" port: 46317 } has no permanent_uuid. Determining permanent_uuid...
I20260806 03:47:14.960789 21777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:14.952105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39860 (local address 127.11.253.253:46317)
0806 03:47:14.952365 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0806 03:47:14.952369 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 03:47:14.952395 (+ 26us) server_negotiation.cc:407] Connection header received
0806 03:47:14.952413 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:47:14.952415 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:47:14.952431 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:47:14.952516 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:47:14.953252 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:14.954089 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:14.954946 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:14.955223 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:14.955280 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:14.957752 (+ 2472us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:47:14.957760 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:47:14.957760 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:47:14.957776 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:47:14.959435 (+ 1659us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:47:14.960027 (+ 592us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:47:14.960029 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:47:14.960029 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:47:14.960079 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:47:14.960278 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:47:14.960279 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:47:14.960279 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:47:14.960449 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:47:14.960579 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0806 03:47:14.960666 (+ 87us) server_negotiation.cc:299] Negotiation successful
0806 03:47:14.960724 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":220,"thread_start_us":122,"threads_started":1}
Aug 06 03:47:14 dist-test-slave-6ntk krb5kdc[20936](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988034, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.253.253@KRBTEST.COM for oryx/127.11.253.253@KRBTEST.COM
I20260806 03:47:14.966544 21773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:14.959806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.253:46317 (local address 127.0.0.1:39876)
0806 03:47:14.959865 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0806 03:47:14.959876 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 03:47:14.959921 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:47:14.960123 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:47:14.960125 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:47:14.960273 (+ 148us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:47:14.960328 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:14.960330 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:14.961204 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:14.961205 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:14.961782 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:14.961784 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:14.962147 (+ 363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:14.962148 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:14.962199 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:14.962499 (+ 300us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:47:14.962502 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:47:14.963548 (+ 1046us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:47:14.965577 (+ 2029us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:47:14.965577 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:47:14.965578 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:47:14.965727 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:47:14.966055 (+ 328us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:47:14.966056 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:47:14.966056 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:47:14.966121 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:47:14.966328 (+ 207us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:47:14.966329 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:47:14.966365 (+ 36us) client_negotiation.cc:769] Sending connection context
0806 03:47:14.966415 (+ 50us) client_negotiation.cc:240] Negotiation successful
0806 03:47:14.966487 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43}
I20260806 03:47:14.966665 21780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:14.959855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39876 (local address 127.11.253.253:46317)
0806 03:47:14.960002 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0806 03:47:14.960004 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:47:14.960012 (+ 8us) server_negotiation.cc:407] Connection header received
0806 03:47:14.960032 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:47:14.960033 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:47:14.960042 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:47:14.960102 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:47:14.960420 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:14.961151 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:14.961847 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:14.962085 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:14.962138 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:14.963620 (+ 1482us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:47:14.963622 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:47:14.963622 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:47:14.963627 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:47:14.965510 (+ 1883us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:47:14.965798 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:47:14.965799 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:47:14.965799 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:47:14.965850 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:47:14.966180 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:47:14.966181 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:47:14.966181 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:47:14.966277 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:47:14.966326 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0806 03:47:14.966429 (+ 103us) server_negotiation.cc:299] Negotiation successful
0806 03:47:14.966532 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":43,"server-negotiator.queue_time_us":125,"thread_start_us":83,"threads_started":1}
I20260806 03:47:14.968009 21765 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 85510334aee348ffbc06964d94b3087b: Bootstrap starting.
I20260806 03:47:14.968632 21765 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 85510334aee348ffbc06964d94b3087b: Neither blocks nor log segments found. Creating new log.
I20260806 03:47:14.968917 21765 log.cc:824] T 00000000000000000000000000000000 P 85510334aee348ffbc06964d94b3087b: Log is configured to *not* fsync() on all Append() calls
I20260806 03:47:14.969630 21765 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 85510334aee348ffbc06964d94b3087b: No bootstrap required, opened a new log
I20260806 03:47:14.971063 21765 raft_consensus.cc:348] T 00000000000000000000000000000000 P 85510334aee348ffbc06964d94b3087b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46d7cfc9ed574f1794039e138d8f6220" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 32833 } } peers { permanent_uuid: "85510334aee348ffbc06964d94b3087b" member_type: VOTER last_known_addr { host: "127.11.253.253" port: 46317 } }
I20260806 03:47:14.971200 21765 raft_consensus.cc:374] T 00000000000000000000000000000000 P 85510334aee348ffbc06964d94b3087b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:47:14.971257 21765 raft_consensus.cc:729] T 00000000000000000000000000000000 P 85510334aee348ffbc06964d94b3087b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85510334aee348ffbc06964d94b3087b, State: Initialized, Role: FOLLOWER
I20260806 03:47:14.971400 21765 consensus_queue.cc:260] T 00000000000000000000000000000000 P 85510334aee348ffbc06964d94b3087b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46d7cfc9ed574f1794039e138d8f6220" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 32833 } } peers { permanent_uuid: "85510334aee348ffbc06964d94b3087b" member_type: VOTER last_known_addr { host: "127.11.253.253" port: 46317 } }
I20260806 03:47:14.971678 21765 sys_catalog.cc:564] T 00000000000000000000000000000000 P 85510334aee348ffbc06964d94b3087b [sys.catalog]: configured and running, proceeding with master startup.
I20260806 03:47:14.971647 21783 sys_catalog.cc:455] T 00000000000000000000000000000000 P 85510334aee348ffbc06964d94b3087b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46d7cfc9ed574f1794039e138d8f6220" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 32833 } } peers { permanent_uuid: "85510334aee348ffbc06964d94b3087b" member_type: VOTER last_known_addr { host: "127.11.253.253" port: 46317 } } }
I20260806 03:47:14.971767 21783 sys_catalog.cc:458] T 00000000000000000000000000000000 P 85510334aee348ffbc06964d94b3087b [sys.catalog]: This master's current role is: FOLLOWER
W20260806 03:47:14.974086 21802 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 85510334aee348ffbc06964d94b3087b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 03:47:14.974169 21802 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 03:47:14.975863 21667 master_runner.cc:428] Detected that this master 85510334aee348ffbc06964d94b3087b is joining an existing cluster
I20260806 03:47:14.975944 21667 master_runner.cc:432] Initiating AddMaster RPC to add 127.11.253.253:46317
I20260806 03:47:14.983742 21812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:14.977109 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.253:46317 (local address 127.0.0.1:39888)
0806 03:47:14.977331 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0806 03:47:14.977340 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 03:47:14.977417 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:47:14.977577 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:47:14.977578 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:47:14.977779 (+ 201us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:47:14.977920 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:14.977924 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:14.978604 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:14.978605 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:14.979187 (+ 582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:14.979189 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:14.979562 (+ 373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:14.979563 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:14.979628 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:14.980365 (+ 737us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:47:14.980369 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:47:14.980814 (+ 445us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:47:14.982691 (+ 1877us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:47:14.982692 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:47:14.982693 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:47:14.982866 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:47:14.983061 (+ 195us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:47:14.983062 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:47:14.983062 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:47:14.983117 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:47:14.983364 (+ 247us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:47:14.983365 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:47:14.983421 (+ 56us) client_negotiation.cc:769] Sending connection context
0806 03:47:14.983510 (+ 89us) client_negotiation.cc:240] Negotiation successful
0806 03:47:14.983617 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":94,"threads_started":1}
I20260806 03:47:14.983798 21780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:14.977206 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39888 (local address 127.11.253.253:46317)
0806 03:47:14.977275 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0806 03:47:14.977278 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 03:47:14.977401 (+ 123us) server_negotiation.cc:407] Connection header received
0806 03:47:14.977482 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:47:14.977483 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:47:14.977493 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:47:14.977555 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:47:14.977996 (+ 441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:14.978528 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:14.979275 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:14.979467 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:14.979548 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:14.980916 (+ 1368us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:47:14.980918 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:47:14.980918 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:47:14.980925 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:47:14.982597 (+ 1672us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:47:14.982944 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:47:14.982945 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:47:14.982946 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:47:14.982996 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:47:14.983182 (+ 186us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:47:14.983183 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:47:14.983183 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:47:14.983301 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:47:14.983372 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0806 03:47:14.983534 (+ 162us) server_negotiation.cc:299] Negotiation successful
0806 03:47:14.983682 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":81,"server-negotiator.queue_time_us":38}
I20260806 03:47:14.984287 21061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:14.977209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42166 (local address 127.11.253.254:32833)
0806 03:47:14.977257 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0806 03:47:14.977259 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:47:14.977577 (+ 318us) server_negotiation.cc:407] Connection header received
0806 03:47:14.977681 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:47:14.977682 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:47:14.977691 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:47:14.977749 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:47:14.978163 (+ 414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:14.978934 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:14.979980 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:14.980224 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:14.980287 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:14.981321 (+ 1034us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:47:14.981323 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:47:14.981323 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:47:14.981330 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:47:14.983011 (+ 1681us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:47:14.983415 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:47:14.983417 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:47:14.983417 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:47:14.983476 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:47:14.983676 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:47:14.983677 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:47:14.983677 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:47:14.983800 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:47:14.983860 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0806 03:47:14.984110 (+ 250us) server_negotiation.cc:299] Negotiation successful
0806 03:47:14.984191 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260806 03:47:14.984629 21814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:14.977109 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:32833 (local address 127.0.0.1:42166)
0806 03:47:14.977520 (+ 411us) negotiation.cc:107] Waiting for socket to connect
0806 03:47:14.977529 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 03:47:14.977600 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:47:14.977753 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:47:14.977754 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:47:14.978023 (+ 269us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:47:14.978077 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:14.978078 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:14.979037 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:14.979038 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:14.979883 (+ 845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:14.979887 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:14.980305 (+ 418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:14.980306 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:14.980372 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:14.980800 (+ 428us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:47:14.980805 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:47:14.981212 (+ 407us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:47:14.983107 (+ 1895us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:47:14.983108 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:47:14.983108 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:47:14.983323 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:47:14.983551 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:47:14.983551 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:47:14.983552 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:47:14.983607 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:47:14.983923 (+ 316us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:47:14.983924 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:47:14.983973 (+ 49us) client_negotiation.cc:769] Sending connection context
0806 03:47:14.984428 (+ 455us) client_negotiation.cc:240] Negotiation successful
0806 03:47:14.984526 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":395,"mutex_wait_us":2,"thread_start_us":101,"threads_started":1}
Aug 06 03:47:14 dist-test-slave-6ntk krb5kdc[20936](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988034, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.253.254@KRBTEST.COM for oryx/127.11.253.253@KRBTEST.COM
I20260806 03:47:14.994227 21780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:14.986689 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39890 (local address 127.11.253.253:46317)
0806 03:47:14.986723 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0806 03:47:14.986724 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 03:47:14.986924 (+ 200us) server_negotiation.cc:407] Connection header received
0806 03:47:14.987054 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:47:14.987055 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:47:14.987064 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:47:14.987118 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:47:14.987627 (+ 509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:14.988170 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:14.989347 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:14.989517 (+ 170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:14.989576 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:14.991741 (+ 2165us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:47:14.991742 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:47:14.991742 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:47:14.991748 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:47:14.993043 (+ 1295us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:47:14.993423 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:47:14.993425 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:47:14.993425 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:47:14.993481 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:47:14.993672 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:47:14.993673 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:47:14.993673 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:47:14.993777 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:47:14.993843 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0806 03:47:14.994062 (+ 219us) server_negotiation.cc:299] Negotiation successful
0806 03:47:14.994137 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260806 03:47:14.994267 21815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:14.986633 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.253:46317 (local address 127.0.0.1:39890)
0806 03:47:14.986812 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0806 03:47:14.986860 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0806 03:47:14.986936 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:47:14.987126 (+ 190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:47:14.987129 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:47:14.987464 (+ 335us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:47:14.987519 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:14.987521 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:14.988228 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:14.988229 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:14.989269 (+ 1040us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:14.989272 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:14.989594 (+ 322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:14.989595 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:14.989686 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:14.990138 (+ 452us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:47:14.990143 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:47:14.991623 (+ 1480us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:47:14.993141 (+ 1518us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:47:14.993148 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:47:14.993150 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:47:14.993349 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:47:14.993551 (+ 202us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:47:14.993552 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:47:14.993553 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:47:14.993612 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:47:14.993849 (+ 237us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:47:14.993856 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:47:14.993926 (+ 70us) client_negotiation.cc:769] Sending connection context
0806 03:47:14.994022 (+ 96us) client_negotiation.cc:240] Negotiation successful
0806 03:47:14.994140 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":76,"threads_started":1}
I20260806 03:47:14.994904 21005 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.11.253.253:46317: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "85510334aee348ffbc06964d94b3087b"
member_type: NON_VOTER
last_known_addr {
host: "127.11.253.253"
port: 46317
}
attrs {
promote: true
}
}
dest_uuid: "46d7cfc9ed574f1794039e138d8f6220"
cas_config_opid_index: -1
I20260806 03:47:14.995261 21005 consensus_queue.cc:237] T 00000000000000000000000000000000 P 46d7cfc9ed574f1794039e138d8f6220 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "46d7cfc9ed574f1794039e138d8f6220" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 32833 } } peers { permanent_uuid: "85510334aee348ffbc06964d94b3087b" member_type: NON_VOTER last_known_addr { host: "127.11.253.253" port: 46317 } attrs { promote: true } }
I20260806 03:47:14.995721 21817 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 46d7cfc9ed574f1794039e138d8f6220 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 85510334aee348ffbc06964d94b3087b (127.11.253.253) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "46d7cfc9ed574f1794039e138d8f6220" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 32833 } } peers { permanent_uuid: "85510334aee348ffbc06964d94b3087b" member_type: NON_VOTER last_known_addr { host: "127.11.253.253" port: 46317 } attrs { promote: true } } }
I20260806 03:47:14.996048 21732 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 85510334aee348ffbc06964d94b3087b [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:47:14.996688 21817 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.11.253.253:46317
I20260806 03:47:14.996788 21817 sys_catalog.cc:455] T 00000000000000000000000000000000 P 46d7cfc9ed574f1794039e138d8f6220 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "46d7cfc9ed574f1794039e138d8f6220" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "46d7cfc9ed574f1794039e138d8f6220" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 32833 } } peers { permanent_uuid: "85510334aee348ffbc06964d94b3087b" member_type: NON_VOTER last_known_addr { host: "127.11.253.253" port: 46317 } attrs { promote: true } } }
I20260806 03:47:14.996861 21817 sys_catalog.cc:458] T 00000000000000000000000000000000 P 46d7cfc9ed574f1794039e138d8f6220 [sys.catalog]: This master's current role is: LEADER
I20260806 03:47:14.996999 21732 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 85510334aee348ffbc06964d94b3087b [term 1 FOLLOWER]: Refusing update from remote peer 46d7cfc9ed574f1794039e138d8f6220: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260806 03:47:14.997020 21667 master.cc:537] Master@127.11.253.253:46317 shutting down...
I20260806 03:47:14.997308 21816 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 46d7cfc9ed574f1794039e138d8f6220 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85510334aee348ffbc06964d94b3087b" member_type: NON_VOTER last_known_addr { host: "127.11.253.253" port: 46317 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
W20260806 03:47:14.998029 20988 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260806 03:47:14.998628 20988 consensus_peers.cc:487] T 00000000000000000000000000000000 P 46d7cfc9ed574f1794039e138d8f6220 -> Peer 85510334aee348ffbc06964d94b3087b (127.11.253.253:46317): Couldn't send request to peer 85510334aee348ffbc06964d94b3087b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260806 03:47:15.000918 21667 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 85510334aee348ffbc06964d94b3087b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 03:47:15.001006 21667 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 85510334aee348ffbc06964d94b3087b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 03:47:15.001065 21667 tablet_replica.cc:330] T 00000000000000000000000000000000 P 85510334aee348ffbc06964d94b3087b: stopping tablet replica
I20260806 03:47:15.013652 21667 master.cc:552] Master@127.11.253.253:46317 shutdown complete.
I20260806 03:47:15.013725 21667 master_runner.cc:305] Clearing existing system tablet
I20260806 03:47:15.013911 21667 fs_manager.cc:359] Metadata directory not provided
I20260806 03:47:15.013965 21667 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:47:15.014611 21667 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.000s sys 0.000s
I20260806 03:47:15.015401 21822 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:47:15.015529 21667 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988004637995-12279-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:47:15.015590 21667 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260806 03:47:15.015631 21667 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988004637995-12279-0/minicluster-data/master-1/data,/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988004637995-12279-0/minicluster-data/master-1/wal
uuid: "85510334aee348ffbc06964d94b3087b"
format_stamp: "Formatted at 2026-08-06 03:47:14 on dist-test-slave-6ntk"
server_key: "0bfe95f85bde7fc653f7d315853c665c"
server_key_iv: "e20d73f2d23538d44c46cf0363283d38"
server_key_version: "encryptionkey@0"
I20260806 03:47:15.016404 21667 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 85510334aee348ffbc06964d94b3087b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 03:47:15.018018 21667 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 85510334aee348ffbc06964d94b3087b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260806 03:47:15.018079 21667 log.cc:1196] T 00000000000000000000000000000000 P 85510334aee348ffbc06964d94b3087b: Deleting WAL directory at /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988004637995-12279-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260806 03:47:15.018304 21667 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 85510334aee348ffbc06964d94b3087b: Deleting consensus metadata
I20260806 03:47:15.018435 21667 master_runner.cc:315] Copying system tablet from 127.11.253.254:32833
I20260806 03:47:15.019073 21667 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 85510334aee348ffbc06964d94b3087b: tablet copy: Beginning tablet copy session from remote peer at address 127.11.253.254:32833
I20260806 03:47:15.026567 21061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:15.019518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42168 (local address 127.11.253.254:32833)
0806 03:47:15.019573 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0806 03:47:15.019575 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:47:15.019663 (+ 88us) server_negotiation.cc:407] Connection header received
0806 03:47:15.019775 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:47:15.019776 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:47:15.019789 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:47:15.019820 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:47:15.020309 (+ 489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:15.020947 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:15.022023 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:15.022553 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:15.022664 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:15.023868 (+ 1204us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:47:15.023869 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:47:15.023869 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:47:15.023877 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:47:15.025424 (+ 1547us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:47:15.025817 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:47:15.025819 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:47:15.025819 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:47:15.025913 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:47:15.026120 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:47:15.026121 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:47:15.026121 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:47:15.026229 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:47:15.026279 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0806 03:47:15.026434 (+ 155us) server_negotiation.cc:299] Negotiation successful
0806 03:47:15.026500 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260806 03:47:15.026654 21829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:15.019421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:32833 (local address 127.0.0.1:42168)
0806 03:47:15.019563 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0806 03:47:15.019595 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0806 03:47:15.019695 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:47:15.019897 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:47:15.019898 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:47:15.020118 (+ 220us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:47:15.020184 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:15.020186 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:15.021052 (+ 866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:15.021053 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:15.021850 (+ 797us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:15.021854 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:15.022752 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:15.022753 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:15.022861 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:15.023344 (+ 483us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:47:15.023350 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:47:15.023770 (+ 420us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:47:15.025504 (+ 1734us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:47:15.025505 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:47:15.025505 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:47:15.025733 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:47:15.025990 (+ 257us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:47:15.025991 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:47:15.025991 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:47:15.026054 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:47:15.026288 (+ 234us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:47:15.026289 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:47:15.026338 (+ 49us) client_negotiation.cc:769] Sending connection context
0806 03:47:15.026418 (+ 80us) client_negotiation.cc:240] Negotiation successful
0806 03:47:15.026536 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":81,"threads_started":1}
I20260806 03:47:15.027055 21031 tablet_copy_service.cc:140] P 46d7cfc9ed574f1794039e138d8f6220: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 85510334aee348ffbc06964d94b3087b ({username='oryx', principal='oryx/127.11.253.253@KRBTEST.COM'} at 127.0.0.1:42168)
I20260806 03:47:15.027208 21031 tablet_copy_service.cc:161] P 46d7cfc9ed574f1794039e138d8f6220: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 85510334aee348ffbc06964d94b3087b at {username='oryx', principal='oryx/127.11.253.253@KRBTEST.COM'} at 127.0.0.1:42168: session id = 85510334aee348ffbc06964d94b3087b-00000000000000000000000000000000
I20260806 03:47:15.028195 21031 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 46d7cfc9ed574f1794039e138d8f6220: Tablet Copy: opened 0 blocks and 1 log segments
I20260806 03:47:15.028927 21667 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260806 03:47:15.030124 21667 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 85510334aee348ffbc06964d94b3087b: tablet copy: Starting download of 0 data blocks...
I20260806 03:47:15.030308 21667 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 85510334aee348ffbc06964d94b3087b: tablet copy: Starting download of 1 WAL segments...
I20260806 03:47:15.031468 21667 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 85510334aee348ffbc06964d94b3087b: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260806 03:47:15.032330 21031 tablet_copy_service.cc:342] P 46d7cfc9ed574f1794039e138d8f6220: Request end of tablet copy session 85510334aee348ffbc06964d94b3087b-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.11.253.253@KRBTEST.COM'} at 127.0.0.1:42168
I20260806 03:47:15.032393 21031 tablet_copy_service.cc:433] P 46d7cfc9ed574f1794039e138d8f6220: ending tablet copy session 85510334aee348ffbc06964d94b3087b-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 85510334aee348ffbc06964d94b3087b
I20260806 03:47:15.035100 21667 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:47:15.036527 21831 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:47:15.036594 21836 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:47:15.036651 21667 server_base.cc:1034] running on GCE node
W20260806 03:47:15.036774 21833 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:47:15.037005 21667 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:47:15.037160 21667 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:47:15.038318 21667 hybrid_clock.cc:648] HybridClock initialized: now 1785988035038304 us; error 31 us; skew 500 ppm
Aug 06 03:47:15 dist-test-slave-6ntk krb5kdc[20936](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988035, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.253.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:47:15.039664 21667 init.cc:377] Logged in from keytab as oryx/127.11.253.253@KRBTEST.COM (short username oryx)
I20260806 03:47:15.039953 21667 webserver.cc:460] Webserver started at http://127.11.253.253:46435/ using document root <none> and password file <none>
I20260806 03:47:15.040076 21667 fs_manager.cc:359] Metadata directory not provided
I20260806 03:47:15.040115 21667 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:47:15.040740 21667 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.000s sys 0.000s
I20260806 03:47:15.041272 21846 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:47:15.041409 21667 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 03:47:15.041460 21667 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988004637995-12279-0/minicluster-data/master-1/data,/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988004637995-12279-0/minicluster-data/master-1/wal
uuid: "85510334aee348ffbc06964d94b3087b"
format_stamp: "Formatted at 2026-08-06 03:47:14 on dist-test-slave-6ntk"
server_key: "0bfe95f85bde7fc653f7d315853c665c"
server_key_iv: "e20d73f2d23538d44c46cf0363283d38"
server_key_version: "encryptionkey@0"
I20260806 03:47:15.041518 21667 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988004637995-12279-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988004637995-12279-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988004637995-12279-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:47:15.059613 21667 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:47:15.061506 21667 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:47:15.065119 21667 rpc_server.cc:307] RPC server started. Bound to: 127.11.253.253:46317
I20260806 03:47:15.065142 21925 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.253.253:46317 every 8 connection(s)
I20260806 03:47:15.065485 21667 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988004637995-12279-0/minicluster-data/master-1/data/info.pb
I20260806 03:47:15.066583 21926 sys_catalog.cc:263] Verifying existing consensus state
I20260806 03:47:15.067166 21926 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 85510334aee348ffbc06964d94b3087b: Bootstrap starting.
I20260806 03:47:15.070533 21926 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 85510334aee348ffbc06964d94b3087b: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 03:47:15.070770 21926 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 85510334aee348ffbc06964d94b3087b: Bootstrap complete.
I20260806 03:47:15.070986 21926 raft_consensus.cc:348] T 00000000000000000000000000000000 P 85510334aee348ffbc06964d94b3087b [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "46d7cfc9ed574f1794039e138d8f6220" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 32833 } } peers { permanent_uuid: "85510334aee348ffbc06964d94b3087b" member_type: NON_VOTER last_known_addr { host: "127.11.253.253" port: 46317 } attrs { promote: true } }
I20260806 03:47:15.071081 21926 raft_consensus.cc:729] T 00000000000000000000000000000000 P 85510334aee348ffbc06964d94b3087b [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 85510334aee348ffbc06964d94b3087b, State: Initialized, Role: LEARNER
I20260806 03:47:15.071121 21926 consensus_queue.cc:260] T 00000000000000000000000000000000 P 85510334aee348ffbc06964d94b3087b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "46d7cfc9ed574f1794039e138d8f6220" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 32833 } } peers { permanent_uuid: "85510334aee348ffbc06964d94b3087b" member_type: NON_VOTER last_known_addr { host: "127.11.253.253" port: 46317 } attrs { promote: true } }
I20260806 03:47:15.071309 21926 sys_catalog.cc:564] T 00000000000000000000000000000000 P 85510334aee348ffbc06964d94b3087b [sys.catalog]: configured and running, proceeding with master startup.
I20260806 03:47:15.071318 21930 sys_catalog.cc:455] T 00000000000000000000000000000000 P 85510334aee348ffbc06964d94b3087b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "46d7cfc9ed574f1794039e138d8f6220" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 32833 } } peers { permanent_uuid: "85510334aee348ffbc06964d94b3087b" member_type: NON_VOTER last_known_addr { host: "127.11.253.253" port: 46317 } attrs { promote: true } } }
I20260806 03:47:15.071482 21930 sys_catalog.cc:458] T 00000000000000000000000000000000 P 85510334aee348ffbc06964d94b3087b [sys.catalog]: This master's current role is: LEARNER
I20260806 03:47:15.072872 21943 catalog_manager.cc:1259] Loaded cluster ID: 21e5bafe25394fe580e3c98318fe43c7
I20260806 03:47:15.072970 21943 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 85510334aee348ffbc06964d94b3087b: loading cluster ID for follower catalog manager: success
I20260806 03:47:15.073961 21943 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 85510334aee348ffbc06964d94b3087b: acquiring CA information for follower catalog manager: success
I20260806 03:47:15.074250 21943 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 85510334aee348ffbc06964d94b3087b: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 03:47:15.120549 21945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:15.113089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39912 (local address 127.11.253.253:46317)
0806 03:47:15.113245 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0806 03:47:15.113247 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:47:15.113257 (+ 10us) server_negotiation.cc:407] Connection header received
0806 03:47:15.113284 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:47:15.113285 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:47:15.113296 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:47:15.113380 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:47:15.114146 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:15.114792 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:15.115859 (+ 1067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:15.116062 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:15.116120 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:15.117637 (+ 1517us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:47:15.117638 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:47:15.117638 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:47:15.117645 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:47:15.119293 (+ 1648us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:47:15.119693 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:47:15.119694 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:47:15.119695 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:47:15.119759 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:47:15.120024 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:47:15.120026 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:47:15.120026 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:47:15.120148 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:47:15.120247 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0806 03:47:15.120369 (+ 122us) server_negotiation.cc:299] Negotiation successful
0806 03:47:15.120440 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":89,"threads_started":1}
WARNING: no policy specified for oryx/127.11.253.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.11.253.252@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.11.253.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988004637995-12279-0/krb5kdc/oryx_127.11.253.252.keytab.
Entry for principal oryx/127.11.253.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988004637995-12279-0/krb5kdc/oryx_127.11.253.252.keytab.
I20260806 03:47:15.149323 12279 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskToVWRe/build/release/bin/kudu
/tmp/dist-test-taskToVWRe/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988004637995-12279-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988004637995-12279-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988004637995-12279-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988004637995-12279-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.253.252:40283
--webserver_interface=127.11.253.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988004637995-12279-0/krb5kdc/oryx_127.11.253.252.keytab
--principal=oryx/127.11.253.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.253.212:45893
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.253.254:32833,127.11.253.253:46317,127.11.253.252:40283
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988004637995-12279-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988004637995-12279-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988004637995-12279-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:47:15.235508 21948 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:47:15.235731 21948 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:47:15.235754 21948 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:47:15.236855 21948 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 03:47:15.236891 21948 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:47:15.236904 21948 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:47:15.236917 21948 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 03:47:15.236928 21948 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 03:47:15.238541 21948 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.253.212:45893
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988004637995-12279-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988004637995-12279-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.11.253.254:32833,127.11.253.253:46317,127.11.253.252:40283
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988004637995-12279-0/krb5kdc/oryx_127.11.253.252.keytab
--principal=oryx/127.11.253.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.253.252:40283
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988004637995-12279-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.11.253.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21948
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988004637995-12279-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:47:15.238746 21948 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:47:15.238942 21948 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:47:15.244618 21958 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:47:15.244630 21957 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:47:15.244812 21961 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:47:15.245039 21948 server_base.cc:1034] running on GCE node
I20260806 03:47:15.245338 21948 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:47:15.245632 21948 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:47:15.246797 21948 hybrid_clock.cc:648] HybridClock initialized: now 1785988035246791 us; error 37 us; skew 500 ppm
Aug 06 03:47:15 dist-test-slave-6ntk krb5kdc[20936](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988035, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.253.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:47:15.249459 21948 init.cc:377] Logged in from keytab as oryx/127.11.253.252@KRBTEST.COM (short username oryx)
I20260806 03:47:15.249958 21948 webserver.cc:460] Webserver started at http://127.11.253.252:34641/ using document root <none> and password file <none>
I20260806 03:47:15.250182 21948 fs_manager.cc:359] Metadata directory not provided
I20260806 03:47:15.250257 21948 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:47:15.250401 21948 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:47:15.251482 21948 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988004637995-12279-0/minicluster-data/master-2/data/instance:
uuid: "492ed5d42bfe4567aa936d6a80e98096"
format_stamp: "Formatted at 2026-08-06 03:47:15 on dist-test-slave-6ntk"
server_key: "70d1edf431d0fb8cfcd4ac11a37ed274"
server_key_iv: "6fa8d3299a687f746904afac0a8f4ae1"
server_key_version: "encryptionkey@0"
I20260806 03:47:15.251904 21948 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988004637995-12279-0/minicluster-data/master-2/wal/instance:
uuid: "492ed5d42bfe4567aa936d6a80e98096"
format_stamp: "Formatted at 2026-08-06 03:47:15 on dist-test-slave-6ntk"
server_key: "70d1edf431d0fb8cfcd4ac11a37ed274"
server_key_iv: "6fa8d3299a687f746904afac0a8f4ae1"
server_key_version: "encryptionkey@0"
I20260806 03:47:15.253603 21948 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 03:47:15.254442 21972 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:47:15.254637 21948 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260806 03:47:15.254737 21948 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988004637995-12279-0/minicluster-data/master-2/data,/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988004637995-12279-0/minicluster-data/master-2/wal
uuid: "492ed5d42bfe4567aa936d6a80e98096"
format_stamp: "Formatted at 2026-08-06 03:47:15 on dist-test-slave-6ntk"
server_key: "70d1edf431d0fb8cfcd4ac11a37ed274"
server_key_iv: "6fa8d3299a687f746904afac0a8f4ae1"
server_key_version: "encryptionkey@0"
I20260806 03:47:15.254819 21948 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988004637995-12279-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988004637995-12279-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988004637995-12279-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:47:15.265951 21948 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:47:15.268299 21948 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:47:15.268419 21948 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:47:15.274497 21948 rpc_server.cc:307] RPC server started. Bound to: 127.11.253.252:40283
I20260806 03:47:15.274542 22047 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.253.252:40283 every 8 connection(s)
I20260806 03:47:15.274904 21948 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988004637995-12279-0/minicluster-data/master-2/data/info.pb
I20260806 03:47:15.276018 22048 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:47:15.277594 22048 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.253.254" port: 32833 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 03:47:15 dist-test-slave-6ntk krb5kdc[20936](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988035, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.253.252@KRBTEST.COM for oryx/127.11.253.254@KRBTEST.COM
I20260806 03:47:15.285187 12279 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskToVWRe/build/release/bin/kudu as pid 21948
I20260806 03:47:15.287487 21061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:15.278440 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42184 (local address 127.11.253.254:32833)
0806 03:47:15.278525 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0806 03:47:15.278527 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:47:15.278696 (+ 169us) server_negotiation.cc:407] Connection header received
0806 03:47:15.279191 (+ 495us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:47:15.279193 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:47:15.279204 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:47:15.279284 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:47:15.279947 (+ 663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:15.280599 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:15.281570 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:15.281788 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:15.281920 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:15.284305 (+ 2385us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:47:15.284307 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:47:15.284307 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:47:15.284315 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:47:15.285862 (+ 1547us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:47:15.286390 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:47:15.286391 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:47:15.286392 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:47:15.286465 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:47:15.286713 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:47:15.286715 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:47:15.286715 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:47:15.286916 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:47:15.287002 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0806 03:47:15.287329 (+ 327us) server_negotiation.cc:299] Negotiation successful
0806 03:47:15.287416 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260806 03:47:15.287748 22054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:15.278332 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:32833 (local address 127.0.0.1:42184)
0806 03:47:15.278571 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0806 03:47:15.278609 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0806 03:47:15.279062 (+ 453us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:47:15.279360 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:47:15.279363 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:47:15.279618 (+ 255us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:47:15.279807 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:15.279811 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:15.280691 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:15.280692 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:15.281475 (+ 783us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:15.281477 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:15.281951 (+ 474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:15.281953 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:15.282083 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:15.282683 (+ 600us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:47:15.282692 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:47:15.284148 (+ 1456us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:47:15.285999 (+ 1851us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:47:15.286006 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:47:15.286009 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:47:15.286316 (+ 307us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:47:15.286547 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:47:15.286547 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:47:15.286547 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:47:15.286648 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:47:15.286989 (+ 341us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:47:15.286994 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:47:15.287156 (+ 162us) client_negotiation.cc:769] Sending connection context
0806 03:47:15.287277 (+ 121us) client_negotiation.cc:240] Negotiation successful
0806 03:47:15.287441 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":161,"threads_started":1}
I20260806 03:47:15.288450 22048 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.253.253" port: 46317 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 03:47:15 dist-test-slave-6ntk krb5kdc[20936](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988034, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.11.253.252@KRBTEST.COM
Aug 06 03:47:15 dist-test-slave-6ntk krb5kdc[20936](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988035, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.253.252@KRBTEST.COM for oryx/127.11.253.253@KRBTEST.COM
I20260806 03:47:15.295920 22055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:15.285769 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44798 (local address 127.11.253.252:40283)
0806 03:47:15.286048 (+ 279us) server_negotiation.cc:206] Beginning negotiation
0806 03:47:15.286054 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 03:47:15.286077 (+ 23us) server_negotiation.cc:407] Connection header received
0806 03:47:15.286099 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:47:15.286101 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:47:15.286114 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:47:15.286196 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:47:15.287063 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:15.288165 (+ 1102us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:15.289115 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:15.289436 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:15.289515 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:15.292868 (+ 3353us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:47:15.292876 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:47:15.292877 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:47:15.292883 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:47:15.294550 (+ 1667us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:47:15.295004 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:47:15.295008 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:47:15.295008 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:47:15.295086 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:47:15.295284 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:47:15.295285 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:47:15.295285 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:47:15.295469 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:47:15.295627 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0806 03:47:15.295721 (+ 94us) server_negotiation.cc:299] Negotiation successful
0806 03:47:15.295782 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":246,"thread_start_us":203,"threads_started":1}
I20260806 03:47:15.296204 21945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:15.288799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39920 (local address 127.11.253.253:46317)
0806 03:47:15.288890 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0806 03:47:15.288893 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 03:47:15.288905 (+ 12us) server_negotiation.cc:407] Connection header received
0806 03:47:15.288940 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:47:15.288941 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:47:15.288954 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:47:15.289017 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:47:15.289325 (+ 308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:15.289979 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:15.291091 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:15.291305 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:15.291402 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:15.293171 (+ 1769us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:47:15.293173 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:47:15.293173 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:47:15.293181 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:47:15.294939 (+ 1758us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:47:15.295339 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:47:15.295340 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:47:15.295341 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:47:15.295391 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:47:15.295583 (+ 192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:47:15.295584 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:47:15.295585 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:47:15.295764 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:47:15.295838 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0806 03:47:15.295988 (+ 150us) server_negotiation.cc:299] Negotiation successful
0806 03:47:15.296098 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260806 03:47:15.296269 22054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:15.288728 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.253:46317 (local address 127.0.0.1:39920)
0806 03:47:15.288778 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0806 03:47:15.288790 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 03:47:15.288857 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:47:15.289022 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:47:15.289023 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:47:15.289171 (+ 148us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:47:15.289236 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:15.289238 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:15.290103 (+ 865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:15.290104 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:15.290998 (+ 894us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:15.291000 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:15.291405 (+ 405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:15.291406 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:15.291496 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:15.291923 (+ 427us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:47:15.291928 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:47:15.293077 (+ 1149us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:47:15.295022 (+ 1945us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:47:15.295022 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:47:15.295023 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:47:15.295264 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:47:15.295454 (+ 190us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:47:15.295455 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:47:15.295455 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:47:15.295518 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:47:15.295836 (+ 318us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:47:15.295837 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:47:15.295875 (+ 38us) client_negotiation.cc:769] Sending connection context
0806 03:47:15.295954 (+ 79us) client_negotiation.cc:240] Negotiation successful
0806 03:47:15.296216 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260806 03:47:15.296855 22048 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.253.252" port: 40283 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 03:47:15 dist-test-slave-6ntk krb5kdc[20936](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988035, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.253.252@KRBTEST.COM for oryx/127.11.253.252@KRBTEST.COM
I20260806 03:47:15.302834 22054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:15.297083 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.252:40283 (local address 127.0.0.1:44812)
0806 03:47:15.297174 (+ 91us) negotiation.cc:107] Waiting for socket to connect
0806 03:47:15.297182 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0806 03:47:15.297220 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:47:15.297323 (+ 103us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:47:15.297323 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:47:15.297441 (+ 118us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:47:15.297468 (+ 27us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:15.297469 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:15.298332 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:15.298333 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:15.298771 (+ 438us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:15.298772 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:15.299105 (+ 333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:15.299106 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:15.299140 (+ 34us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:15.299491 (+ 351us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:47:15.299495 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:47:15.300538 (+ 1043us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:47:15.302117 (+ 1579us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:47:15.302117 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:47:15.302117 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:47:15.302264 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:47:15.302419 (+ 155us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:47:15.302420 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:47:15.302420 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:47:15.302467 (+ 47us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:47:15.302631 (+ 164us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:47:15.302631 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:47:15.302662 (+ 31us) client_negotiation.cc:769] Sending connection context
0806 03:47:15.302711 (+ 49us) client_negotiation.cc:240] Negotiation successful
0806 03:47:15.302779 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":79}
I20260806 03:47:15.303011 22055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:15.297135 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44812 (local address 127.11.253.252:40283)
0806 03:47:15.297188 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0806 03:47:15.297190 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:47:15.297211 (+ 21us) server_negotiation.cc:407] Connection header received
0806 03:47:15.297255 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:47:15.297256 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:47:15.297265 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:47:15.297319 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:47:15.297724 (+ 405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:15.298265 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:15.298844 (+ 579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:15.299042 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:15.299096 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:15.300623 (+ 1527us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:47:15.300624 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:47:15.300624 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:47:15.300630 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:47:15.302046 (+ 1416us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:47:15.302319 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:47:15.302319 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:47:15.302319 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:47:15.302367 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:47:15.302504 (+ 137us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:47:15.302505 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:47:15.302505 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:47:15.302586 (+ 81us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:47:15.302630 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0806 03:47:15.302725 (+ 95us) server_negotiation.cc:299] Negotiation successful
0806 03:47:15.302822 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":41,"server-negotiator.queue_time_us":35}
I20260806 03:47:15.304292 22048 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 492ed5d42bfe4567aa936d6a80e98096: Bootstrap starting.
I20260806 03:47:15.305128 22048 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 492ed5d42bfe4567aa936d6a80e98096: Neither blocks nor log segments found. Creating new log.
I20260806 03:47:15.305366 22048 log.cc:824] T 00000000000000000000000000000000 P 492ed5d42bfe4567aa936d6a80e98096: Log is configured to *not* fsync() on all Append() calls
I20260806 03:47:15.306192 22048 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 492ed5d42bfe4567aa936d6a80e98096: No bootstrap required, opened a new log
I20260806 03:47:15.307624 22048 raft_consensus.cc:348] T 00000000000000000000000000000000 P 492ed5d42bfe4567aa936d6a80e98096 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46d7cfc9ed574f1794039e138d8f6220" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 32833 } } peers { permanent_uuid: "85510334aee348ffbc06964d94b3087b" member_type: VOTER last_known_addr { host: "127.11.253.253" port: 46317 } } peers { permanent_uuid: "492ed5d42bfe4567aa936d6a80e98096" member_type: VOTER last_known_addr { host: "127.11.253.252" port: 40283 } }
I20260806 03:47:15.307746 22048 raft_consensus.cc:374] T 00000000000000000000000000000000 P 492ed5d42bfe4567aa936d6a80e98096 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:47:15.307770 22048 raft_consensus.cc:729] T 00000000000000000000000000000000 P 492ed5d42bfe4567aa936d6a80e98096 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 492ed5d42bfe4567aa936d6a80e98096, State: Initialized, Role: FOLLOWER
I20260806 03:47:15.307898 22048 consensus_queue.cc:260] T 00000000000000000000000000000000 P 492ed5d42bfe4567aa936d6a80e98096 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46d7cfc9ed574f1794039e138d8f6220" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 32833 } } peers { permanent_uuid: "85510334aee348ffbc06964d94b3087b" member_type: VOTER last_known_addr { host: "127.11.253.253" port: 46317 } } peers { permanent_uuid: "492ed5d42bfe4567aa936d6a80e98096" member_type: VOTER last_known_addr { host: "127.11.253.252" port: 40283 } }
I20260806 03:47:15.308149 22057 sys_catalog.cc:455] T 00000000000000000000000000000000 P 492ed5d42bfe4567aa936d6a80e98096 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46d7cfc9ed574f1794039e138d8f6220" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 32833 } } peers { permanent_uuid: "85510334aee348ffbc06964d94b3087b" member_type: VOTER last_known_addr { host: "127.11.253.253" port: 46317 } } peers { permanent_uuid: "492ed5d42bfe4567aa936d6a80e98096" member_type: VOTER last_known_addr { host: "127.11.253.252" port: 40283 } } }
I20260806 03:47:15.308203 22048 sys_catalog.cc:564] T 00000000000000000000000000000000 P 492ed5d42bfe4567aa936d6a80e98096 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 03:47:15.308252 22057 sys_catalog.cc:458] T 00000000000000000000000000000000 P 492ed5d42bfe4567aa936d6a80e98096 [sys.catalog]: This master's current role is: FOLLOWER
W20260806 03:47:15.309986 22075 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 492ed5d42bfe4567aa936d6a80e98096: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 03:47:15.310046 22075 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 03:47:15.312271 21948 master_runner.cc:428] Detected that this master 492ed5d42bfe4567aa936d6a80e98096 is joining an existing cluster
I20260806 03:47:15.312321 21948 master_runner.cc:432] Initiating AddMaster RPC to add 127.11.253.252:40283
I20260806 03:47:15.320250 21061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:15.313537 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42196 (local address 127.11.253.254:32833)
0806 03:47:15.313587 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0806 03:47:15.313590 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 03:47:15.313749 (+ 159us) server_negotiation.cc:407] Connection header received
0806 03:47:15.313840 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:47:15.313841 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:47:15.313851 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:47:15.313936 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:47:15.314356 (+ 420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:15.315096 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:15.316246 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:15.316493 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:15.316545 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:15.317589 (+ 1044us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:47:15.317590 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:47:15.317591 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:47:15.317597 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:47:15.319094 (+ 1497us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:47:15.319461 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:47:15.319463 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:47:15.319463 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:47:15.319522 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:47:15.319718 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:47:15.319719 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:47:15.319719 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:47:15.319831 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:47:15.319908 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0806 03:47:15.320085 (+ 177us) server_negotiation.cc:299] Negotiation successful
0806 03:47:15.320158 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260806 03:47:15.320266 22085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:15.313478 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:32833 (local address 127.0.0.1:42196)
0806 03:47:15.313683 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0806 03:47:15.313695 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 03:47:15.313776 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:47:15.314000 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:47:15.314001 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:47:15.314200 (+ 199us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:47:15.314281 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:15.314283 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:15.315181 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:15.315182 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:15.316155 (+ 973us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:15.316159 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:15.316587 (+ 428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:15.316588 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:15.316659 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:15.317071 (+ 412us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:47:15.317076 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:47:15.317499 (+ 423us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:47:15.319197 (+ 1698us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:47:15.319198 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:47:15.319198 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:47:15.319375 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:47:15.319588 (+ 213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:47:15.319588 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:47:15.319589 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:47:15.319648 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:47:15.319903 (+ 255us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:47:15.319904 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:47:15.319971 (+ 67us) client_negotiation.cc:769] Sending connection context
0806 03:47:15.320046 (+ 75us) client_negotiation.cc:240] Negotiation successful
0806 03:47:15.320152 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":178,"mutex_wait_us":29,"thread_start_us":102,"threads_started":1}
I20260806 03:47:15.324359 22055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:15.313950 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44828 (local address 127.11.253.252:40283)
0806 03:47:15.313986 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0806 03:47:15.313988 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:47:15.318047 (+ 4059us) server_negotiation.cc:407] Connection header received
0806 03:47:15.318110 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:47:15.318111 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:47:15.318122 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:47:15.318185 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:47:15.318553 (+ 368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:15.319233 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:15.320240 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:15.320527 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:15.320568 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:15.321823 (+ 1255us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:47:15.321824 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:47:15.321824 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:47:15.321830 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:47:15.323266 (+ 1436us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:47:15.323647 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:47:15.323647 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:47:15.323648 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:47:15.323695 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:47:15.323896 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:47:15.323896 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:47:15.323896 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:47:15.324012 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:47:15.324064 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0806 03:47:15.324233 (+ 169us) server_negotiation.cc:299] Negotiation successful
0806 03:47:15.324300 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":21}
I20260806 03:47:15.324605 22084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:15.313706 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.252:40283 (local address 127.0.0.1:44828)
0806 03:47:15.317986 (+ 4280us) negotiation.cc:107] Waiting for socket to connect
0806 03:47:15.317997 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 03:47:15.318056 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:47:15.318200 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:47:15.318200 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:47:15.318401 (+ 201us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:47:15.318466 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:15.318468 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:15.319321 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:15.319321 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:15.320149 (+ 828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:15.320151 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:15.320836 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:15.320837 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:15.320897 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:15.321278 (+ 381us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:47:15.321284 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:47:15.321718 (+ 434us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:47:15.323351 (+ 1633us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:47:15.323352 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:47:15.323352 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:47:15.323572 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:47:15.323759 (+ 187us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:47:15.323760 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:47:15.323760 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:47:15.323837 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:47:15.324078 (+ 241us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:47:15.324079 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:47:15.324139 (+ 60us) client_negotiation.cc:769] Sending connection context
0806 03:47:15.324218 (+ 79us) client_negotiation.cc:240] Negotiation successful
0806 03:47:15.324373 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4252,"mutex_wait_us":50,"thread_start_us":123,"threads_started":1}
I20260806 03:47:15.333809 21945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:15.313726 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39936 (local address 127.11.253.253:46317)
0806 03:47:15.313764 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0806 03:47:15.313765 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 03:47:15.314008 (+ 243us) server_negotiation.cc:407] Connection header received
0806 03:47:15.314103 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:47:15.314104 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:47:15.314114 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:47:15.314175 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:47:15.314544 (+ 369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:15.315226 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:15.315967 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:15.316173 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:15.316255 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:15.330891 (+ 14636us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:47:15.330894 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:47:15.330894 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:47:15.330904 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:47:15.332601 (+ 1697us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:47:15.332996 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:47:15.332997 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:47:15.332997 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:47:15.333063 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:47:15.333276 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:47:15.333277 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:47:15.333277 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:47:15.333405 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:47:15.333475 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0806 03:47:15.333637 (+ 162us) server_negotiation.cc:299] Negotiation successful
0806 03:47:15.333716 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 03:47:15.333827 22086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:15.313548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.253:46317 (local address 127.0.0.1:39936)
0806 03:47:15.313943 (+ 395us) negotiation.cc:107] Waiting for socket to connect
0806 03:47:15.313953 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 03:47:15.314035 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:47:15.314178 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:47:15.314178 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:47:15.314390 (+ 212us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:47:15.314461 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:15.314463 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:15.315305 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:15.315306 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:15.315863 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:15.315865 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:15.316593 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:15.316594 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:15.316646 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:15.330341 (+ 13695us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:47:15.330347 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:47:15.330764 (+ 417us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:47:15.332703 (+ 1939us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:47:15.332704 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:47:15.332704 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:47:15.332914 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:47:15.333133 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:47:15.333134 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:47:15.333134 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:47:15.333212 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:47:15.333479 (+ 267us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:47:15.333480 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:47:15.333524 (+ 44us) client_negotiation.cc:769] Sending connection context
0806 03:47:15.333598 (+ 74us) client_negotiation.cc:240] Negotiation successful
0806 03:47:15.333711 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":379,"mutex_wait_us":1,"thread_start_us":83,"threads_started":1}
Aug 06 03:47:15 dist-test-slave-6ntk krb5kdc[20936](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988034, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.253.254@KRBTEST.COM for oryx/127.11.253.252@KRBTEST.COM
I20260806 03:47:15.339205 21815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:15.331507 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.252:40283 (local address 127.0.0.1:44830)
0806 03:47:15.331563 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0806 03:47:15.331575 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 03:47:15.331618 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:47:15.331948 (+ 330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:47:15.331950 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:47:15.332087 (+ 137us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:47:15.332131 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:15.332133 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:15.332765 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:15.332767 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:15.333694 (+ 927us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:15.333697 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:15.334250 (+ 553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:15.334251 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:15.334331 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:15.334794 (+ 463us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:47:15.334799 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:47:15.336446 (+ 1647us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:47:15.338251 (+ 1805us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:47:15.338252 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:47:15.338252 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:47:15.338425 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:47:15.338640 (+ 215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:47:15.338640 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:47:15.338641 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:47:15.338693 (+ 52us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:47:15.338968 (+ 275us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:47:15.338969 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:47:15.339007 (+ 38us) client_negotiation.cc:769] Sending connection context
0806 03:47:15.339054 (+ 47us) client_negotiation.cc:240] Negotiation successful
0806 03:47:15.339137 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260806 03:47:15.339278 22055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:15.331561 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44830 (local address 127.11.253.252:40283)
0806 03:47:15.331597 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0806 03:47:15.331599 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:47:15.331607 (+ 8us) server_negotiation.cc:407] Connection header received
0806 03:47:15.331701 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:47:15.331702 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:47:15.331712 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:47:15.331744 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:47:15.332208 (+ 464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:15.332693 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:15.333817 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:15.334161 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:15.334288 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:15.336534 (+ 2246us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:47:15.336536 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:47:15.336536 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:47:15.336544 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:47:15.338170 (+ 1626us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:47:15.338501 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:47:15.338502 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:47:15.338503 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:47:15.338567 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:47:15.338757 (+ 190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:47:15.338757 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:47:15.338758 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:47:15.338897 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:47:15.338964 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0806 03:47:15.339097 (+ 133us) server_negotiation.cc:299] Negotiation successful
0806 03:47:15.339180 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260806 03:47:15.339862 21005 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.11.253.252:40283: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "492ed5d42bfe4567aa936d6a80e98096"
member_type: NON_VOTER
last_known_addr {
host: "127.11.253.252"
port: 40283
}
attrs {
promote: true
}
}
dest_uuid: "46d7cfc9ed574f1794039e138d8f6220"
cas_config_opid_index: 5
I20260806 03:47:15.340135 21005 consensus_queue.cc:237] T 00000000000000000000000000000000 P 46d7cfc9ed574f1794039e138d8f6220 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "46d7cfc9ed574f1794039e138d8f6220" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 32833 } } peers { permanent_uuid: "85510334aee348ffbc06964d94b3087b" member_type: NON_VOTER last_known_addr { host: "127.11.253.253" port: 46317 } attrs { promote: true } } peers { permanent_uuid: "492ed5d42bfe4567aa936d6a80e98096" member_type: NON_VOTER last_known_addr { host: "127.11.253.252" port: 40283 } attrs { promote: true } }
I20260806 03:47:15.340479 21816 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 46d7cfc9ed574f1794039e138d8f6220 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 492ed5d42bfe4567aa936d6a80e98096 (127.11.253.252) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "46d7cfc9ed574f1794039e138d8f6220" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 32833 } } peers { permanent_uuid: "85510334aee348ffbc06964d94b3087b" member_type: NON_VOTER last_known_addr { host: "127.11.253.253" port: 46317 } attrs { promote: true } } peers { permanent_uuid: "492ed5d42bfe4567aa936d6a80e98096" member_type: NON_VOTER last_known_addr { host: "127.11.253.252" port: 40283 } attrs { promote: true } } }
I20260806 03:47:15.341229 22011 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 492ed5d42bfe4567aa936d6a80e98096 [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:47:15.341636 21816 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.11.253.252:40283
I20260806 03:47:15.341734 21816 sys_catalog.cc:455] T 00000000000000000000000000000000 P 46d7cfc9ed574f1794039e138d8f6220 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "46d7cfc9ed574f1794039e138d8f6220" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "46d7cfc9ed574f1794039e138d8f6220" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 32833 } } peers { permanent_uuid: "85510334aee348ffbc06964d94b3087b" member_type: NON_VOTER last_known_addr { host: "127.11.253.253" port: 46317 } attrs { promote: true } } peers { permanent_uuid: "492ed5d42bfe4567aa936d6a80e98096" member_type: NON_VOTER last_known_addr { host: "127.11.253.252" port: 40283 } attrs { promote: true } } }
I20260806 03:47:15.341817 21816 sys_catalog.cc:458] T 00000000000000000000000000000000 P 46d7cfc9ed574f1794039e138d8f6220 [sys.catalog]: This master's current role is: LEADER
I20260806 03:47:15.342195 21948 master.cc:537] Master@127.11.253.252:40283 shutting down...
I20260806 03:47:15.342260 22011 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 492ed5d42bfe4567aa936d6a80e98096 [term 1 FOLLOWER]: Refusing update from remote peer 46d7cfc9ed574f1794039e138d8f6220: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260806 03:47:15.342547 21817 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 46d7cfc9ed574f1794039e138d8f6220 [LEADER]: Connected to new peer: Peer: permanent_uuid: "492ed5d42bfe4567aa936d6a80e98096" member_type: NON_VOTER last_known_addr { host: "127.11.253.252" port: 40283 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
W20260806 03:47:15.343392 20984 consensus_peers.cc:487] T 00000000000000000000000000000000 P 46d7cfc9ed574f1794039e138d8f6220 -> Peer 492ed5d42bfe4567aa936d6a80e98096 (127.11.253.252:40283): Couldn't send request to peer 492ed5d42bfe4567aa936d6a80e98096. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260806 03:47:15.345465 21948 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 492ed5d42bfe4567aa936d6a80e98096 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 03:47:15.345578 21948 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 492ed5d42bfe4567aa936d6a80e98096 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 03:47:15.345638 21948 tablet_replica.cc:330] T 00000000000000000000000000000000 P 492ed5d42bfe4567aa936d6a80e98096: stopping tablet replica
I20260806 03:47:15.346246 21945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:15.340636 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39944 (local address 127.11.253.253:46317)
0806 03:47:15.340757 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0806 03:47:15.340759 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:47:15.340766 (+ 7us) server_negotiation.cc:407] Connection header received
0806 03:47:15.340786 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:47:15.340787 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:47:15.340815 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:47:15.340881 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 03:47:15.341057 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:15.342016 (+ 959us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:15.345002 (+ 2986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:15.345668 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:15.345820 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:15.345941 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0806 03:47:15.346130 (+ 189us) server_negotiation.cc:299] Negotiation successful
0806 03:47:15.346173 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102}
I20260806 03:47:15.346251 21815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:15.340591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.253:46317 (local address 127.0.0.1:39944)
0806 03:47:15.340633 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0806 03:47:15.340643 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 03:47:15.340681 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:47:15.340936 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:47:15.340938 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:47:15.340939 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 03:47:15.340979 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:15.340981 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:15.342116 (+ 1135us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:15.342118 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:15.344906 (+ 2788us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:15.344909 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:15.345957 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:15.345958 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:15.346022 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:15.346046 (+ 24us) client_negotiation.cc:769] Sending connection context
0806 03:47:15.346114 (+ 68us) client_negotiation.cc:240] Negotiation successful
0806 03:47:15.346156 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28}
I20260806 03:47:15.346946 21890 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 85510334aee348ffbc06964d94b3087b [term 1 LEARNER]: Refusing update from remote peer 46d7cfc9ed574f1794039e138d8f6220: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260806 03:47:15.347155 21817 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 46d7cfc9ed574f1794039e138d8f6220 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85510334aee348ffbc06964d94b3087b" member_type: NON_VOTER last_known_addr { host: "127.11.253.253" port: 46317 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260806 03:47:15.347767 21890 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 85510334aee348ffbc06964d94b3087b [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 492ed5d42bfe4567aa936d6a80e98096 (127.11.253.252) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "46d7cfc9ed574f1794039e138d8f6220" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 32833 } } peers { permanent_uuid: "85510334aee348ffbc06964d94b3087b" member_type: NON_VOTER last_known_addr { host: "127.11.253.253" port: 46317 } attrs { promote: true } } peers { permanent_uuid: "492ed5d42bfe4567aa936d6a80e98096" member_type: NON_VOTER last_known_addr { host: "127.11.253.252" port: 40283 } attrs { promote: true } } }
I20260806 03:47:15.348785 21948 master.cc:552] Master@127.11.253.252:40283 shutdown complete.
I20260806 03:47:15.348837 21948 master_runner.cc:305] Clearing existing system tablet
I20260806 03:47:15.348976 21948 fs_manager.cc:359] Metadata directory not provided
I20260806 03:47:15.349020 21948 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:47:15.349239 21930 sys_catalog.cc:455] T 00000000000000000000000000000000 P 85510334aee348ffbc06964d94b3087b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 46d7cfc9ed574f1794039e138d8f6220. Latest consensus state: current_term: 1 leader_uuid: "46d7cfc9ed574f1794039e138d8f6220" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "46d7cfc9ed574f1794039e138d8f6220" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 32833 } } peers { permanent_uuid: "85510334aee348ffbc06964d94b3087b" member_type: NON_VOTER last_known_addr { host: "127.11.253.253" port: 46317 } attrs { promote: true } } peers { permanent_uuid: "492ed5d42bfe4567aa936d6a80e98096" member_type: NON_VOTER last_known_addr { host: "127.11.253.252" port: 40283 } attrs { promote: true } } }
I20260806 03:47:15.349334 21930 sys_catalog.cc:458] T 00000000000000000000000000000000 P 85510334aee348ffbc06964d94b3087b [sys.catalog]: This master's current role is: LEARNER
I20260806 03:47:15.349375 21930 sys_catalog.cc:455] T 00000000000000000000000000000000 P 85510334aee348ffbc06964d94b3087b [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "46d7cfc9ed574f1794039e138d8f6220" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "46d7cfc9ed574f1794039e138d8f6220" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 32833 } } peers { permanent_uuid: "85510334aee348ffbc06964d94b3087b" member_type: NON_VOTER last_known_addr { host: "127.11.253.253" port: 46317 } attrs { promote: true } } peers { permanent_uuid: "492ed5d42bfe4567aa936d6a80e98096" member_type: NON_VOTER last_known_addr { host: "127.11.253.252" port: 40283 } attrs { promote: true } } }
I20260806 03:47:15.349421 21930 sys_catalog.cc:458] T 00000000000000000000000000000000 P 85510334aee348ffbc06964d94b3087b [sys.catalog]: This master's current role is: LEARNER
I20260806 03:47:15.349411 21817 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 46d7cfc9ed574f1794039e138d8f6220: attempting to promote NON_VOTER 85510334aee348ffbc06964d94b3087b to VOTER
I20260806 03:47:15.349623 21817 consensus_queue.cc:237] T 00000000000000000000000000000000 P 46d7cfc9ed574f1794039e138d8f6220 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "46d7cfc9ed574f1794039e138d8f6220" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 32833 } } peers { permanent_uuid: "85510334aee348ffbc06964d94b3087b" member_type: VOTER last_known_addr { host: "127.11.253.253" port: 46317 } attrs { promote: false } } peers { permanent_uuid: "492ed5d42bfe4567aa936d6a80e98096" member_type: NON_VOTER last_known_addr { host: "127.11.253.252" port: 40283 } attrs { promote: true } }
I20260806 03:47:15.349800 21948 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 03:47:15.350148 21890 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 85510334aee348ffbc06964d94b3087b [term 1 LEARNER]: Refusing update from remote peer 46d7cfc9ed574f1794039e138d8f6220: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260806 03:47:15.350167 21815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:15.350053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.252:40283 (local address 127.0.0.1:44856)
0806 03:47:15.350094 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0806 03:47:15.350135 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.253.252:40283: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
I20260806 03:47:15.350329 21816 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 46d7cfc9ed574f1794039e138d8f6220 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85510334aee348ffbc06964d94b3087b" member_type: VOTER last_known_addr { host: "127.11.253.253" port: 46317 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260806 03:47:15.350461 21815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:15.350365 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.252:40283 (local address 127.0.0.1:44870)
0806 03:47:15.350395 (+ 30us) negotiation.cc:107] Waiting for socket to connect
0806 03:47:15.350432 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.253.252:40283: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":19}
W20260806 03:47:15.350518 20984 consensus_peers.cc:487] T 00000000000000000000000000000000 P 46d7cfc9ed574f1794039e138d8f6220 -> Peer 492ed5d42bfe4567aa936d6a80e98096 (127.11.253.252:40283): Couldn't send request to peer 492ed5d42bfe4567aa936d6a80e98096. Status: Network error: Client connection negotiation failed: client connection to 127.11.253.252:40283: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260806 03:47:15.350735 22093 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:47:15.350874 21948 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988004637995-12279-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:47:15.350945 21948 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260806 03:47:15.350977 21948 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988004637995-12279-0/minicluster-data/master-2/data,/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988004637995-12279-0/minicluster-data/master-2/wal
uuid: "492ed5d42bfe4567aa936d6a80e98096"
format_stamp: "Formatted at 2026-08-06 03:47:15 on dist-test-slave-6ntk"
server_key: "70d1edf431d0fb8cfcd4ac11a37ed274"
server_key_iv: "6fa8d3299a687f746904afac0a8f4ae1"
server_key_version: "encryptionkey@0"
I20260806 03:47:15.351183 21817 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 46d7cfc9ed574f1794039e138d8f6220 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 85510334aee348ffbc06964d94b3087b (127.11.253.253) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "46d7cfc9ed574f1794039e138d8f6220" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 32833 } } peers { permanent_uuid: "85510334aee348ffbc06964d94b3087b" member_type: VOTER last_known_addr { host: "127.11.253.253" port: 46317 } attrs { promote: false } } peers { permanent_uuid: "492ed5d42bfe4567aa936d6a80e98096" member_type: NON_VOTER last_known_addr { host: "127.11.253.252" port: 40283 } attrs { promote: true } } }
I20260806 03:47:15.351809 21948 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 492ed5d42bfe4567aa936d6a80e98096: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 03:47:15.352166 21817 sys_catalog.cc:455] T 00000000000000000000000000000000 P 46d7cfc9ed574f1794039e138d8f6220 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "46d7cfc9ed574f1794039e138d8f6220" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "46d7cfc9ed574f1794039e138d8f6220" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 32833 } } peers { permanent_uuid: "85510334aee348ffbc06964d94b3087b" member_type: VOTER last_known_addr { host: "127.11.253.253" port: 46317 } attrs { promote: false } } peers { permanent_uuid: "492ed5d42bfe4567aa936d6a80e98096" member_type: NON_VOTER last_known_addr { host: "127.11.253.252" port: 40283 } attrs { promote: true } } }
I20260806 03:47:15.352234 21817 sys_catalog.cc:458] T 00000000000000000000000000000000 P 46d7cfc9ed574f1794039e138d8f6220 [sys.catalog]: This master's current role is: LEADER
I20260806 03:47:15.352602 21816 sys_catalog.cc:455] T 00000000000000000000000000000000 P 46d7cfc9ed574f1794039e138d8f6220 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "46d7cfc9ed574f1794039e138d8f6220" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "46d7cfc9ed574f1794039e138d8f6220" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 32833 } } peers { permanent_uuid: "85510334aee348ffbc06964d94b3087b" member_type: VOTER last_known_addr { host: "127.11.253.253" port: 46317 } attrs { promote: false } } peers { permanent_uuid: "492ed5d42bfe4567aa936d6a80e98096" member_type: NON_VOTER last_known_addr { host: "127.11.253.252" port: 40283 } attrs { promote: true } } }
I20260806 03:47:15.352677 21816 sys_catalog.cc:458] T 00000000000000000000000000000000 P 46d7cfc9ed574f1794039e138d8f6220 [sys.catalog]: This master's current role is: LEADER
I20260806 03:47:15.353047 21890 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 85510334aee348ffbc06964d94b3087b [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 85510334aee348ffbc06964d94b3087b (127.11.253.253) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "46d7cfc9ed574f1794039e138d8f6220" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 32833 } } peers { permanent_uuid: "85510334aee348ffbc06964d94b3087b" member_type: VOTER last_known_addr { host: "127.11.253.253" port: 46317 } attrs { promote: false } } peers { permanent_uuid: "492ed5d42bfe4567aa936d6a80e98096" member_type: NON_VOTER last_known_addr { host: "127.11.253.252" port: 40283 } attrs { promote: true } } }
I20260806 03:47:15.353451 21948 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 492ed5d42bfe4567aa936d6a80e98096: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260806 03:47:15.353536 21948 log.cc:1196] T 00000000000000000000000000000000 P 492ed5d42bfe4567aa936d6a80e98096: Deleting WAL directory at /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988004637995-12279-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260806 03:47:15.353775 21948 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 492ed5d42bfe4567aa936d6a80e98096: Deleting consensus metadata
I20260806 03:47:15.353971 21948 master_runner.cc:315] Copying system tablet from 127.11.253.254:32833
I20260806 03:47:15.354203 22088 sys_catalog.cc:455] T 00000000000000000000000000000000 P 85510334aee348ffbc06964d94b3087b [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "46d7cfc9ed574f1794039e138d8f6220" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "46d7cfc9ed574f1794039e138d8f6220" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 32833 } } peers { permanent_uuid: "85510334aee348ffbc06964d94b3087b" member_type: VOTER last_known_addr { host: "127.11.253.253" port: 46317 } attrs { promote: false } } peers { permanent_uuid: "492ed5d42bfe4567aa936d6a80e98096" member_type: NON_VOTER last_known_addr { host: "127.11.253.252" port: 40283 } attrs { promote: true } } }
I20260806 03:47:15.354305 22088 sys_catalog.cc:458] T 00000000000000000000000000000000 P 85510334aee348ffbc06964d94b3087b [sys.catalog]: This master's current role is: FOLLOWER
I20260806 03:47:15.354651 21948 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 492ed5d42bfe4567aa936d6a80e98096: tablet copy: Beginning tablet copy session from remote peer at address 127.11.253.254:32833
I20260806 03:47:15.360890 21061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:15.355024 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42200 (local address 127.11.253.254:32833)
0806 03:47:15.355064 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0806 03:47:15.355066 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:47:15.355189 (+ 123us) server_negotiation.cc:407] Connection header received
0806 03:47:15.355266 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:47:15.355267 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:47:15.355277 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:47:15.355333 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:47:15.355713 (+ 380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:15.356188 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:15.356984 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:15.357175 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:15.357255 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:15.358282 (+ 1027us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:47:15.358284 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:47:15.358284 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:47:15.358292 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:47:15.359738 (+ 1446us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:47:15.360094 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:47:15.360095 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:47:15.360095 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:47:15.360153 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:47:15.360359 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:47:15.360360 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:47:15.360360 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:47:15.360467 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:47:15.360538 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0806 03:47:15.360706 (+ 168us) server_negotiation.cc:299] Negotiation successful
0806 03:47:15.360788 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 03:47:15.360915 22104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:15.354978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:32833 (local address 127.0.0.1:42200)
0806 03:47:15.355112 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0806 03:47:15.355122 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 03:47:15.355205 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:47:15.355338 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:47:15.355339 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:47:15.355526 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:47:15.355648 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:15.355651 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:15.356275 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:15.356276 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:15.356906 (+ 630us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:15.356908 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:15.357258 (+ 350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:15.357259 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:15.357330 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:15.357846 (+ 516us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:47:15.357850 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:47:15.358200 (+ 350us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:47:15.359824 (+ 1624us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:47:15.359826 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:47:15.359826 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:47:15.360021 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:47:15.360224 (+ 203us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:47:15.360225 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:47:15.360225 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:47:15.360290 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:47:15.360542 (+ 252us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:47:15.360543 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:47:15.360590 (+ 47us) client_negotiation.cc:769] Sending connection context
0806 03:47:15.360677 (+ 87us) client_negotiation.cc:240] Negotiation successful
0806 03:47:15.360790 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":83,"threads_started":1}
I20260806 03:47:15.361335 21031 tablet_copy_service.cc:140] P 46d7cfc9ed574f1794039e138d8f6220: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 492ed5d42bfe4567aa936d6a80e98096 ({username='oryx', principal='oryx/127.11.253.252@KRBTEST.COM'} at 127.0.0.1:42200)
I20260806 03:47:15.361415 21031 tablet_copy_service.cc:161] P 46d7cfc9ed574f1794039e138d8f6220: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 492ed5d42bfe4567aa936d6a80e98096 at {username='oryx', principal='oryx/127.11.253.252@KRBTEST.COM'} at 127.0.0.1:42200: session id = 492ed5d42bfe4567aa936d6a80e98096-00000000000000000000000000000000
I20260806 03:47:15.361855 21031 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 46d7cfc9ed574f1794039e138d8f6220: Tablet Copy: opened 0 blocks and 1 log segments
I20260806 03:47:15.362439 21948 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260806 03:47:15.363390 21948 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 492ed5d42bfe4567aa936d6a80e98096: tablet copy: Starting download of 0 data blocks...
I20260806 03:47:15.363559 21948 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 492ed5d42bfe4567aa936d6a80e98096: tablet copy: Starting download of 1 WAL segments...
I20260806 03:47:15.364454 21948 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 492ed5d42bfe4567aa936d6a80e98096: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260806 03:47:15.365226 21031 tablet_copy_service.cc:342] P 46d7cfc9ed574f1794039e138d8f6220: Request end of tablet copy session 492ed5d42bfe4567aa936d6a80e98096-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.11.253.252@KRBTEST.COM'} at 127.0.0.1:42200
I20260806 03:47:15.365290 21031 tablet_copy_service.cc:433] P 46d7cfc9ed574f1794039e138d8f6220: ending tablet copy session 492ed5d42bfe4567aa936d6a80e98096-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 492ed5d42bfe4567aa936d6a80e98096
I20260806 03:47:15.368541 21948 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:47:15.369845 22108 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:47:15.369830 22109 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:47:15.374945 22113 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:47:15.375072 21948 server_base.cc:1034] running on GCE node
I20260806 03:47:15.375174 21948 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:47:15.375335 21948 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:47:15.376456 21948 hybrid_clock.cc:648] HybridClock initialized: now 1785988035376448 us; error 25 us; skew 500 ppm
Aug 06 03:47:15 dist-test-slave-6ntk krb5kdc[20936](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988035, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.253.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:47:15.378083 21948 init.cc:377] Logged in from keytab as oryx/127.11.253.252@KRBTEST.COM (short username oryx)
I20260806 03:47:15.378392 21948 webserver.cc:460] Webserver started at http://127.11.253.252:43761/ using document root <none> and password file <none>
I20260806 03:47:15.378528 21948 fs_manager.cc:359] Metadata directory not provided
I20260806 03:47:15.378569 21948 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:47:15.379387 21948 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 03:47:15.380064 22128 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:47:15.380216 21948 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 03:47:15.380278 21948 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988004637995-12279-0/minicluster-data/master-2/data,/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988004637995-12279-0/minicluster-data/master-2/wal
uuid: "492ed5d42bfe4567aa936d6a80e98096"
format_stamp: "Formatted at 2026-08-06 03:47:15 on dist-test-slave-6ntk"
server_key: "70d1edf431d0fb8cfcd4ac11a37ed274"
server_key_iv: "6fa8d3299a687f746904afac0a8f4ae1"
server_key_version: "encryptionkey@0"
I20260806 03:47:15.380344 21948 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988004637995-12279-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988004637995-12279-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988004637995-12279-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:47:15.397459 21948 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:47:15.399166 21948 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:47:15.405756 21948 rpc_server.cc:307] RPC server started. Bound to: 127.11.253.252:40283
I20260806 03:47:15.406034 22188 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.253.252:40283 every 8 connection(s)
I20260806 03:47:15.406288 21948 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988004637995-12279-0/minicluster-data/master-2/data/info.pb
I20260806 03:47:15.407215 22190 sys_catalog.cc:263] Verifying existing consensus state
I20260806 03:47:15.407626 22190 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 492ed5d42bfe4567aa936d6a80e98096: Bootstrap starting.
I20260806 03:47:15.410188 22190 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 492ed5d42bfe4567aa936d6a80e98096: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 03:47:15.410445 22190 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 492ed5d42bfe4567aa936d6a80e98096: Bootstrap complete.
I20260806 03:47:15.410650 22190 raft_consensus.cc:348] T 00000000000000000000000000000000 P 492ed5d42bfe4567aa936d6a80e98096 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "46d7cfc9ed574f1794039e138d8f6220" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 32833 } } peers { permanent_uuid: "85510334aee348ffbc06964d94b3087b" member_type: VOTER last_known_addr { host: "127.11.253.253" port: 46317 } attrs { promote: false } } peers { permanent_uuid: "492ed5d42bfe4567aa936d6a80e98096" member_type: NON_VOTER last_known_addr { host: "127.11.253.252" port: 40283 } attrs { promote: true } }
I20260806 03:47:15.410742 22190 raft_consensus.cc:729] T 00000000000000000000000000000000 P 492ed5d42bfe4567aa936d6a80e98096 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 492ed5d42bfe4567aa936d6a80e98096, State: Initialized, Role: LEARNER
I20260806 03:47:15.410810 22190 consensus_queue.cc:260] T 00000000000000000000000000000000 P 492ed5d42bfe4567aa936d6a80e98096 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "46d7cfc9ed574f1794039e138d8f6220" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 32833 } } peers { permanent_uuid: "85510334aee348ffbc06964d94b3087b" member_type: VOTER last_known_addr { host: "127.11.253.253" port: 46317 } attrs { promote: false } } peers { permanent_uuid: "492ed5d42bfe4567aa936d6a80e98096" member_type: NON_VOTER last_known_addr { host: "127.11.253.252" port: 40283 } attrs { promote: true } }
I20260806 03:47:15.411016 22190 sys_catalog.cc:564] T 00000000000000000000000000000000 P 492ed5d42bfe4567aa936d6a80e98096 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 03:47:15.411012 22193 sys_catalog.cc:455] T 00000000000000000000000000000000 P 492ed5d42bfe4567aa936d6a80e98096 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "46d7cfc9ed574f1794039e138d8f6220" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 32833 } } peers { permanent_uuid: "85510334aee348ffbc06964d94b3087b" member_type: VOTER last_known_addr { host: "127.11.253.253" port: 46317 } attrs { promote: false } } peers { permanent_uuid: "492ed5d42bfe4567aa936d6a80e98096" member_type: NON_VOTER last_known_addr { host: "127.11.253.252" port: 40283 } attrs { promote: true } } }
I20260806 03:47:15.411132 22193 sys_catalog.cc:458] T 00000000000000000000000000000000 P 492ed5d42bfe4567aa936d6a80e98096 [sys.catalog]: This master's current role is: LEARNER
I20260806 03:47:15.412590 22208 catalog_manager.cc:1259] Loaded cluster ID: 21e5bafe25394fe580e3c98318fe43c7
I20260806 03:47:15.412751 22208 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 492ed5d42bfe4567aa936d6a80e98096: loading cluster ID for follower catalog manager: success
I20260806 03:47:15.413712 22208 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 492ed5d42bfe4567aa936d6a80e98096: acquiring CA information for follower catalog manager: success
I20260806 03:47:15.414014 22208 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 492ed5d42bfe4567aa936d6a80e98096: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 03:47:15.455685 22209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:15.448106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44890 (local address 127.11.253.252:40283)
0806 03:47:15.448287 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0806 03:47:15.448289 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:47:15.448296 (+ 7us) server_negotiation.cc:407] Connection header received
0806 03:47:15.448320 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:47:15.448320 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:47:15.448330 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:47:15.448410 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:47:15.449019 (+ 609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:15.449837 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:15.450595 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:15.450810 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:15.450888 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:15.452229 (+ 1341us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:47:15.452231 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:47:15.452231 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:47:15.452239 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:47:15.454090 (+ 1851us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:47:15.454503 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:47:15.454505 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:47:15.454505 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:47:15.454583 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:47:15.454823 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:47:15.454824 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:47:15.454825 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:47:15.454988 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:47:15.455122 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0806 03:47:15.455268 (+ 146us) server_negotiation.cc:299] Negotiation successful
0806 03:47:15.455545 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":91,"threads_started":1}
I20260806 03:47:15.456347 12279 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskToVWRe/build/release/bin/kudu with pid 21096
I20260806 03:47:15.477429 12279 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskToVWRe/build/release/bin/kudu
/tmp/dist-test-taskToVWRe/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988004637995-12279-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988004637995-12279-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988004637995-12279-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988004637995-12279-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.253.193:34025
--local_ip_for_outbound_sockets=127.11.253.193
--tserver_master_addrs=127.11.253.254:32833
--webserver_port=35453
--webserver_interface=127.11.253.193
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988004637995-12279-0/krb5kdc/oryx_127.11.253.193.keytab
--principal=oryx/127.11.253.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.253.212:45893
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.11.253.254:32833,127.11.253.253:46317
--tserver_master_addrs=127.11.253.254:32833,127.11.253.253:46317,127.11.253.252:40283 with env {KRB5CCNAME=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988004637995-12279-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988004637995-12279-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988004637995-12279-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:47:15.568723 22210 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:47:15.569017 22210 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:47:15.569059 22210 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:47:15.570299 22210 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:47:15.570361 22210 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:47:15.570407 22210 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.253.193
I20260806 03:47:15.572161 22210 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.253.212:45893
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988004637995-12279-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988004637995-12279-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988004637995-12279-0/krb5kdc/oryx_127.11.253.193.keytab
--principal=oryx/127.11.253.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.253.193:34025
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988004637995-12279-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.253.193
--webserver_port=35453
--tserver_master_addrs=127.11.253.254:32833,127.11.253.253:46317,127.11.253.252:40283
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22210
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.253.193
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988004637995-12279-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:47:15.572446 22210 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:47:15.572751 22210 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:47:15.578049 22218 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:47:15.578217 22210 server_base.cc:1034] running on GCE node
W20260806 03:47:15.578038 22217 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:47:15.578038 22220 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:47:15.578658 22210 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:47:15.578925 22210 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:47:15.580076 22210 hybrid_clock.cc:648] HybridClock initialized: now 1785988035580059 us; error 38 us; skew 500 ppm
Aug 06 03:47:15 dist-test-slave-6ntk krb5kdc[20936](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988035, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.253.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:47:15.582758 22210 init.cc:377] Logged in from keytab as oryx/127.11.253.193@KRBTEST.COM (short username oryx)
I20260806 03:47:15.583204 22210 webserver.cc:460] Webserver started at http://127.11.253.193:35453/ using document root <none> and password file <none>
I20260806 03:47:15.583443 22210 fs_manager.cc:359] Metadata directory not provided
I20260806 03:47:15.583500 22210 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:47:15.584884 22210 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 03:47:15.585471 22231 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:47:15.585659 22210 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 03:47:15.585767 22210 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988004637995-12279-0/minicluster-data/ts-0/data,/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988004637995-12279-0/minicluster-data/ts-0/wal
uuid: "21d27ac5b0aa4681b44bc0482b668d54"
format_stamp: "Formatted at 2026-08-06 03:47:14 on dist-test-slave-6ntk"
server_key: "4ca1b394c8127be478d17a105e99a79b"
server_key_iv: "2b4a03b4ec313f9ba6c43b4f53089da7"
server_key_version: "encryptionkey@0"
I20260806 03:47:15.586169 22210 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988004637995-12279-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988004637995-12279-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988004637995-12279-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:47:15.596529 22210 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:47:15.598953 22210 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:47:15.599128 22210 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:47:15.599359 22210 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 03:47:15.599673 22210 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 03:47:15.599707 22210 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 03:47:15.599730 22210 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 03:47:15.599742 22210 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 03:47:15.605343 22210 rpc_server.cc:307] RPC server started. Bound to: 127.11.253.193:34025
I20260806 03:47:15.605428 22396 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.253.193:34025 every 8 connection(s)
I20260806 03:47:15.605744 22210 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988004637995-12279-0/minicluster-data/ts-0/data/info.pb
Aug 06 03:47:15 dist-test-slave-6ntk krb5kdc[20936](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988035, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.253.193@KRBTEST.COM for oryx/127.11.253.253@KRBTEST.COM
Aug 06 03:47:15 dist-test-slave-6ntk krb5kdc[20936](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988035, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.253.193@KRBTEST.COM for oryx/127.11.253.254@KRBTEST.COM
I20260806 03:47:15.613792 12279 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskToVWRe/build/release/bin/kudu as pid 22210
I20260806 03:47:15.613987 12279 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskToVWRe/build/release/bin/kudu with pid 21285
Aug 06 03:47:15 dist-test-slave-6ntk krb5kdc[20936](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988035, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.253.193@KRBTEST.COM for oryx/127.11.253.252@KRBTEST.COM
I20260806 03:47:15.620087 21945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:15.606604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.253.193:54477 (local address 127.11.253.253:46317)
0806 03:47:15.606651 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0806 03:47:15.606653 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:47:15.606981 (+ 328us) server_negotiation.cc:407] Connection header received
0806 03:47:15.607987 (+ 1006us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:47:15.607989 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:47:15.608001 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:47:15.608045 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:47:15.608837 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:15.609341 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:15.610400 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:15.610612 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:15.610686 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:15.613635 (+ 2949us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:47:15.613637 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:47:15.613638 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:47:15.613646 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:47:15.618333 (+ 4687us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:47:15.618879 (+ 546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:47:15.618881 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:47:15.618882 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:47:15.618959 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:47:15.619364 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:47:15.619365 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:47:15.619365 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:47:15.619565 (+ 200us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:47:15.619620 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0806 03:47:15.619867 (+ 247us) server_negotiation.cc:299] Negotiation successful
0806 03:47:15.619953 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 03:47:15.620209 22209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:15.606531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.253.193:58749 (local address 127.11.253.252:40283)
0806 03:47:15.606577 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0806 03:47:15.606579 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:47:15.607005 (+ 426us) server_negotiation.cc:407] Connection header received
0806 03:47:15.608047 (+ 1042us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:47:15.608048 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:47:15.608064 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:47:15.608097 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:47:15.608860 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:15.609634 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:15.612436 (+ 2802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:15.612762 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:15.612849 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:15.615193 (+ 2344us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:47:15.615194 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:47:15.615195 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:47:15.615202 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:47:15.618665 (+ 3463us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:47:15.619061 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:47:15.619062 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:47:15.619062 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:47:15.619114 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:47:15.619321 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:47:15.619321 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:47:15.619322 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:47:15.619452 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:47:15.619490 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0806 03:47:15.619919 (+ 429us) server_negotiation.cc:299] Negotiation successful
0806 03:47:15.620039 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 03:47:15.620651 22406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:15.606539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.253:46317 (local address 127.11.253.193:54477)
0806 03:47:15.606836 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0806 03:47:15.606882 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0806 03:47:15.607883 (+ 1001us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:47:15.608108 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:47:15.608117 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:47:15.608545 (+ 428us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:47:15.608740 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:15.608745 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:15.609447 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:15.609448 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:15.610316 (+ 868us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:15.610318 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:15.610694 (+ 376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:15.610695 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:15.610973 (+ 278us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:15.611534 (+ 561us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:47:15.611542 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:47:15.613509 (+ 1967us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:47:15.618422 (+ 4913us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:47:15.618430 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:47:15.618432 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:47:15.618758 (+ 326us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:47:15.619053 (+ 295us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:47:15.619054 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:47:15.619055 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:47:15.619262 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:47:15.619678 (+ 416us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:47:15.619679 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:47:15.619720 (+ 41us) client_negotiation.cc:769] Sending connection context
0806 03:47:15.620173 (+ 453us) client_negotiation.cc:240] Negotiation successful
0806 03:47:15.620352 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":191,"threads_started":1}
I20260806 03:47:15.620673 22407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:15.606539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.252:40283 (local address 127.11.253.193:58749)
0806 03:47:15.606917 (+ 378us) negotiation.cc:107] Waiting for socket to connect
0806 03:47:15.606926 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 03:47:15.607916 (+ 990us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:47:15.608190 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:47:15.608191 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:47:15.608599 (+ 408us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:47:15.608742 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:15.608748 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:15.609729 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:15.609730 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:15.612347 (+ 2617us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:15.612350 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:15.612849 (+ 499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:15.612850 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:15.612900 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:15.613676 (+ 776us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:47:15.613681 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:47:15.615100 (+ 1419us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:47:15.618779 (+ 3679us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:47:15.618780 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:47:15.618781 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:47:15.618986 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:47:15.619174 (+ 188us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:47:15.619175 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:47:15.619175 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:47:15.619261 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:47:15.619532 (+ 271us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:47:15.619537 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:47:15.619707 (+ 170us) client_negotiation.cc:769] Sending connection context
0806 03:47:15.620325 (+ 618us) client_negotiation.cc:240] Negotiation successful
0806 03:47:15.620467 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":352,"mutex_wait_us":134,"thread_start_us":201,"threads_started":1}
I20260806 03:47:15.623215 22397 heartbeater.cc:347] Connected to a master server at 127.11.253.252:40283
I20260806 03:47:15.623500 22397 heartbeater.cc:464] Registering TS with master...
I20260806 03:47:15.623731 22397 heartbeater.cc:511] Master 127.11.253.252:40283 requested a full tablet report, sending...
I20260806 03:47:15.624228 22401 heartbeater.cc:347] Connected to a master server at 127.11.253.253:46317
I20260806 03:47:15.624280 22401 heartbeater.cc:464] Registering TS with master...
I20260806 03:47:15.624351 22401 heartbeater.cc:511] Master 127.11.253.253:46317 requested a full tablet report, sending...
I20260806 03:47:15.624629 22148 ts_manager.cc:194] Registered new tserver with Master: 21d27ac5b0aa4681b44bc0482b668d54 (127.11.253.193:34025)
I20260806 03:47:15.625125 21871 ts_manager.cc:194] Registered new tserver with Master: 21d27ac5b0aa4681b44bc0482b668d54 (127.11.253.193:34025)
I20260806 03:47:15.628312 21061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:15.607141 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.253.193:37185 (local address 127.11.253.254:32833)
0806 03:47:15.607205 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0806 03:47:15.607207 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:47:15.607220 (+ 13us) server_negotiation.cc:407] Connection header received
0806 03:47:15.608043 (+ 823us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:47:15.608045 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:47:15.608059 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:47:15.608095 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:47:15.608863 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:15.609622 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:15.610809 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:15.611119 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:15.611191 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:15.623837 (+ 12646us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:47:15.623840 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:47:15.623840 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:47:15.623850 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:47:15.627122 (+ 3272us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:47:15.627567 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:47:15.627568 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:47:15.627568 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:47:15.627626 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:47:15.627826 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:47:15.627826 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:47:15.627827 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:47:15.627960 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:47:15.628016 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0806 03:47:15.628177 (+ 161us) server_negotiation.cc:299] Negotiation successful
0806 03:47:15.628245 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260806 03:47:15.629051 22409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:15.606968 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:32833 (local address 127.11.253.193:37185)
0806 03:47:15.607087 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0806 03:47:15.607096 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 03:47:15.607919 (+ 823us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:47:15.608190 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:47:15.608191 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:47:15.608592 (+ 401us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:47:15.608742 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:15.608748 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:15.609752 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:15.609753 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:15.610714 (+ 961us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:15.610716 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:15.611472 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:15.611473 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:15.611539 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:15.611957 (+ 418us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:47:15.611962 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:47:15.623687 (+ 11725us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:47:15.627216 (+ 3529us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:47:15.627218 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:47:15.627218 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:47:15.627484 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:47:15.627684 (+ 200us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:47:15.627685 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:47:15.627685 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:47:15.627756 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:47:15.628024 (+ 268us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:47:15.628025 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:47:15.628077 (+ 52us) client_negotiation.cc:769] Sending connection context
0806 03:47:15.628151 (+ 74us) client_negotiation.cc:240] Negotiation successful
0806 03:47:15.628283 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":101,"mutex_wait_us":24,"thread_start_us":65,"threads_started":1}
I20260806 03:47:15.629468 12279 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskToVWRe/build/release/bin/kudu
/tmp/dist-test-taskToVWRe/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988004637995-12279-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988004637995-12279-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988004637995-12279-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988004637995-12279-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.253.194:45307
--local_ip_for_outbound_sockets=127.11.253.194
--tserver_master_addrs=127.11.253.254:32833
--webserver_port=46449
--webserver_interface=127.11.253.194
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988004637995-12279-0/krb5kdc/oryx_127.11.253.194.keytab
--principal=oryx/127.11.253.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.253.212:45893
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.11.253.254:32833,127.11.253.253:46317
--tserver_master_addrs=127.11.253.254:32833,127.11.253.253:46317,127.11.253.252:40283 with env {KRB5CCNAME=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988004637995-12279-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988004637995-12279-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988004637995-12279-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 03:47:15.629477 22399 heartbeater.cc:347] Connected to a master server at 127.11.253.254:32833
I20260806 03:47:15.629550 22399 heartbeater.cc:464] Registering TS with master...
I20260806 03:47:15.629632 22399 heartbeater.cc:511] Master 127.11.253.254:32833 requested a full tablet report, sending...
I20260806 03:47:15.629942 21005 ts_manager.cc:194] Re-registered known tserver with Master: 21d27ac5b0aa4681b44bc0482b668d54 (127.11.253.193:34025)
I20260806 03:47:15.630455 21005 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.11.253.193@KRBTEST.COM'} at 127.11.253.193:37185
W20260806 03:47:15.713229 22411 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:47:15.713428 22411 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:47:15.713450 22411 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:47:15.714548 22411 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:47:15.714586 22411 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:47:15.714612 22411 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.253.194
I20260806 03:47:15.716003 22411 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.253.212:45893
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988004637995-12279-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988004637995-12279-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988004637995-12279-0/krb5kdc/oryx_127.11.253.194.keytab
--principal=oryx/127.11.253.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.253.194:45307
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988004637995-12279-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.253.194
--webserver_port=46449
--tserver_master_addrs=127.11.253.254:32833,127.11.253.253:46317,127.11.253.252:40283
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22411
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.253.194
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988004637995-12279-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:47:15.716276 22411 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:47:15.716507 22411 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:47:15.721692 22425 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:47:15.721706 22421 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:47:15.721841 22411 server_base.cc:1034] running on GCE node
W20260806 03:47:15.721692 22418 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:47:15.722259 22411 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:47:15.722479 22411 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:47:15.723634 22411 hybrid_clock.cc:648] HybridClock initialized: now 1785988035723619 us; error 30 us; skew 500 ppm
Aug 06 03:47:15 dist-test-slave-6ntk krb5kdc[20936](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988035, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.253.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:47:15.726408 22411 init.cc:377] Logged in from keytab as oryx/127.11.253.194@KRBTEST.COM (short username oryx)
I20260806 03:47:15.726886 22411 webserver.cc:460] Webserver started at http://127.11.253.194:46449/ using document root <none> and password file <none>
I20260806 03:47:15.727136 22411 fs_manager.cc:359] Metadata directory not provided
I20260806 03:47:15.727212 22411 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:47:15.728585 22411 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.001s
I20260806 03:47:15.729225 22439 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:47:15.729423 22411 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260806 03:47:15.729528 22411 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988004637995-12279-0/minicluster-data/ts-1/data,/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988004637995-12279-0/minicluster-data/ts-1/wal
uuid: "0fa8b770e150486aae31b4df6ad38a6e"
format_stamp: "Formatted at 2026-08-06 03:47:14 on dist-test-slave-6ntk"
server_key: "cbc31c6e3f3148dd1569cd5d80a5e891"
server_key_iv: "d07befe1b47a3bd7b2c79c2ba7fd8abb"
server_key_version: "encryptionkey@0"
I20260806 03:47:15.729900 22411 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988004637995-12279-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988004637995-12279-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988004637995-12279-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:47:15.740120 22411 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:47:15.742197 22411 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:47:15.742347 22411 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:47:15.742583 22411 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 03:47:15.742916 22411 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 03:47:15.742950 22411 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 03:47:15.743000 22411 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 03:47:15.743026 22411 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 03:47:15.750324 22411 rpc_server.cc:307] RPC server started. Bound to: 127.11.253.194:45307
I20260806 03:47:15.750398 22596 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.253.194:45307 every 8 connection(s)
I20260806 03:47:15.750777 22411 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988004637995-12279-0/minicluster-data/ts-1/data/info.pb
I20260806 03:47:15.754667 12279 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskToVWRe/build/release/bin/kudu as pid 22411
I20260806 03:47:15.754845 12279 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskToVWRe/build/release/bin/kudu with pid 21472
Aug 06 03:47:15 dist-test-slave-6ntk krb5kdc[20936](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988035, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.253.194@KRBTEST.COM for oryx/127.11.253.253@KRBTEST.COM
Aug 06 03:47:15 dist-test-slave-6ntk krb5kdc[20936](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988035, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.253.194@KRBTEST.COM for oryx/127.11.253.252@KRBTEST.COM
I20260806 03:47:15.764874 21945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:15.751579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.253.194:41517 (local address 127.11.253.253:46317)
0806 03:47:15.751636 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0806 03:47:15.751638 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:47:15.751850 (+ 212us) server_negotiation.cc:407] Connection header received
0806 03:47:15.752535 (+ 685us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:47:15.752537 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:47:15.752550 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:47:15.752584 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:47:15.753370 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:15.754179 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:15.755952 (+ 1773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:15.756158 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:15.756193 (+ 35us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:15.761210 (+ 5017us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:47:15.761211 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:47:15.761211 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:47:15.761220 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:47:15.763363 (+ 2143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:47:15.763804 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:47:15.763806 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:47:15.763806 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:47:15.763883 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:47:15.764191 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:47:15.764193 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:47:15.764193 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:47:15.764331 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:47:15.764413 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0806 03:47:15.764677 (+ 264us) server_negotiation.cc:299] Negotiation successful
0806 03:47:15.764760 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260806 03:47:15.765071 22209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:15.751524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.253.194:54097 (local address 127.11.253.252:40283)
0806 03:47:15.751644 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0806 03:47:15.751647 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 03:47:15.751844 (+ 197us) server_negotiation.cc:407] Connection header received
0806 03:47:15.752519 (+ 675us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:47:15.752521 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:47:15.752534 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:47:15.752566 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:47:15.753406 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:15.754189 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:15.758250 (+ 4061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:15.758551 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:15.758597 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:15.761491 (+ 2894us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:47:15.761492 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:47:15.761493 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:47:15.761500 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:47:15.763776 (+ 2276us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:47:15.764152 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:47:15.764153 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:47:15.764154 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:47:15.764217 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:47:15.764430 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:47:15.764431 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:47:15.764432 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:47:15.764579 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:47:15.764621 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0806 03:47:15.764817 (+ 196us) server_negotiation.cc:299] Negotiation successful
0806 03:47:15.764936 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260806 03:47:15.765110 22606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:15.751536 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.253:46317 (local address 127.11.253.194:41517)
0806 03:47:15.751726 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0806 03:47:15.751767 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 03:47:15.752400 (+ 633us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:47:15.752657 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:47:15.752660 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:47:15.753042 (+ 382us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:47:15.753269 (+ 227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:15.753274 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:15.755299 (+ 2025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:15.755300 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:15.755867 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:15.755869 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:15.756268 (+ 399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:15.756269 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:15.756403 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:15.756875 (+ 472us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:47:15.756884 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:47:15.759299 (+ 2415us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:47:15.763468 (+ 4169us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:47:15.763476 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:47:15.763479 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:47:15.763713 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:47:15.763947 (+ 234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:47:15.763948 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:47:15.763949 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:47:15.764123 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:47:15.764405 (+ 282us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:47:15.764413 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:47:15.764550 (+ 137us) client_negotiation.cc:769] Sending connection context
0806 03:47:15.764657 (+ 107us) client_negotiation.cc:240] Negotiation successful
0806 03:47:15.764820 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"mutex_wait_us":4,"spinlock_wait_cycles":8064,"thread_start_us":117,"threads_started":1}
I20260806 03:47:15.765425 22605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:15.751537 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.252:40283 (local address 127.11.253.194:54097)
0806 03:47:15.751723 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0806 03:47:15.751781 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0806 03:47:15.752409 (+ 628us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:47:15.752654 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:47:15.752659 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:47:15.753095 (+ 436us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:47:15.753270 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:15.753275 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:15.754261 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:15.754262 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:15.755216 (+ 954us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:15.755218 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:15.758677 (+ 3459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:15.758678 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:15.758743 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:15.759437 (+ 694us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:47:15.759442 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:47:15.761107 (+ 1665us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:47:15.763867 (+ 2760us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:47:15.763868 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:47:15.763869 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:47:15.764070 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:47:15.764283 (+ 213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:47:15.764284 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:47:15.764285 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:47:15.764349 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:47:15.764666 (+ 317us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:47:15.764667 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:47:15.764711 (+ 44us) client_negotiation.cc:769] Sending connection context
0806 03:47:15.765162 (+ 451us) client_negotiation.cc:240] Negotiation successful
0806 03:47:15.765279 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":94,"threads_started":1}
I20260806 03:47:15.765769 22600 heartbeater.cc:347] Connected to a master server at 127.11.253.252:40283
I20260806 03:47:15.765893 22600 heartbeater.cc:464] Registering TS with master...
I20260806 03:47:15.765998 22603 heartbeater.cc:347] Connected to a master server at 127.11.253.253:46317
I20260806 03:47:15.766108 22600 heartbeater.cc:511] Master 127.11.253.252:40283 requested a full tablet report, sending...
I20260806 03:47:15.766170 22603 heartbeater.cc:464] Registering TS with master...
I20260806 03:47:15.766314 22603 heartbeater.cc:511] Master 127.11.253.253:46317 requested a full tablet report, sending...
I20260806 03:47:15.766649 21871 ts_manager.cc:194] Registered new tserver with Master: 0fa8b770e150486aae31b4df6ad38a6e (127.11.253.194:45307)
I20260806 03:47:15.767051 22148 ts_manager.cc:194] Registered new tserver with Master: 0fa8b770e150486aae31b4df6ad38a6e (127.11.253.194:45307)
Aug 06 03:47:15 dist-test-slave-6ntk krb5kdc[20936](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988035, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.253.194@KRBTEST.COM for oryx/127.11.253.254@KRBTEST.COM
I20260806 03:47:15.768989 12279 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskToVWRe/build/release/bin/kudu
/tmp/dist-test-taskToVWRe/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988004637995-12279-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988004637995-12279-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988004637995-12279-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988004637995-12279-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.253.195:33553
--local_ip_for_outbound_sockets=127.11.253.195
--tserver_master_addrs=127.11.253.254:32833
--webserver_port=34977
--webserver_interface=127.11.253.195
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988004637995-12279-0/krb5kdc/oryx_127.11.253.195.keytab
--principal=oryx/127.11.253.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.253.212:45893
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.11.253.254:32833,127.11.253.253:46317
--tserver_master_addrs=127.11.253.254:32833,127.11.253.253:46317,127.11.253.252:40283 with env {KRB5CCNAME=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988004637995-12279-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988004637995-12279-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988004637995-12279-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 03:47:15.772517 21061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:15.751902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.253.194:48075 (local address 127.11.253.254:32833)
0806 03:47:15.751995 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0806 03:47:15.751998 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 03:47:15.752112 (+ 114us) server_negotiation.cc:407] Connection header received
0806 03:47:15.752505 (+ 393us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:47:15.752507 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:47:15.752523 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:47:15.752584 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:47:15.753388 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:15.753980 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:15.766476 (+ 12496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:15.766820 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:15.766877 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:15.769569 (+ 2692us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:47:15.769571 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:47:15.769571 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:47:15.769579 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:47:15.771349 (+ 1770us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:47:15.771682 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:47:15.771684 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:47:15.771684 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:47:15.771770 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:47:15.771955 (+ 185us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:47:15.771956 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:47:15.771956 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:47:15.772103 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:47:15.772173 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0806 03:47:15.772281 (+ 108us) server_negotiation.cc:299] Negotiation successful
0806 03:47:15.772375 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260806 03:47:15.772557 22607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:15.751840 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:32833 (local address 127.11.253.194:48075)
0806 03:47:15.752045 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0806 03:47:15.752053 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0806 03:47:15.752408 (+ 355us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:47:15.752654 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:47:15.752659 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:47:15.753201 (+ 542us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:47:15.753270 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:15.753276 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:15.754084 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:15.754085 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:15.766362 (+ 12277us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:15.766365 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:15.767212 (+ 847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:15.767213 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:15.767297 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:15.767650 (+ 353us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:47:15.767656 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:47:15.769332 (+ 1676us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:47:15.771438 (+ 2106us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:47:15.771439 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:47:15.771439 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:47:15.771607 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:47:15.771856 (+ 249us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:47:15.771856 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:47:15.771856 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:47:15.771909 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:47:15.772165 (+ 256us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:47:15.772166 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:47:15.772201 (+ 35us) client_negotiation.cc:769] Sending connection context
0806 03:47:15.772285 (+ 84us) client_negotiation.cc:240] Negotiation successful
0806 03:47:15.772423 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":186,"mutex_wait_us":3,"thread_start_us":62,"threads_started":1}
I20260806 03:47:15.772891 22602 heartbeater.cc:347] Connected to a master server at 127.11.253.254:32833
I20260806 03:47:15.772941 22602 heartbeater.cc:464] Registering TS with master...
I20260806 03:47:15.773006 22602 heartbeater.cc:511] Master 127.11.253.254:32833 requested a full tablet report, sending...
I20260806 03:47:15.773234 21005 ts_manager.cc:194] Re-registered known tserver with Master: 0fa8b770e150486aae31b4df6ad38a6e (127.11.253.194:45307)
I20260806 03:47:15.773758 21005 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.11.253.194@KRBTEST.COM'} at 127.11.253.194:48075
W20260806 03:47:15.856165 22608 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:47:15.856405 22608 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:47:15.856446 22608 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:47:15.857651 22608 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:47:15.857713 22608 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:47:15.857755 22608 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.253.195
I20260806 03:47:15.859310 22608 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.253.212:45893
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988004637995-12279-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988004637995-12279-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988004637995-12279-0/krb5kdc/oryx_127.11.253.195.keytab
--principal=oryx/127.11.253.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.253.195:33553
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988004637995-12279-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.253.195
--webserver_port=34977
--tserver_master_addrs=127.11.253.254:32833,127.11.253.253:46317,127.11.253.252:40283
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22608
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.253.195
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988004637995-12279-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:47:15.859584 22608 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:47:15.859828 22608 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:47:15.865127 22621 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:47:15.865147 22608 server_base.cc:1034] running on GCE node
W20260806 03:47:15.865128 22619 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:47:15.865197 22617 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:47:15.865584 22608 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:47:15.865842 22608 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:47:15.867048 22608 hybrid_clock.cc:648] HybridClock initialized: now 1785988035867028 us; error 39 us; skew 500 ppm
Aug 06 03:47:15 dist-test-slave-6ntk krb5kdc[20936](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988035, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.253.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:47:15.869753 22608 init.cc:377] Logged in from keytab as oryx/127.11.253.195@KRBTEST.COM (short username oryx)
I20260806 03:47:15.870280 22608 webserver.cc:460] Webserver started at http://127.11.253.195:34977/ using document root <none> and password file <none>
I20260806 03:47:15.870522 22608 fs_manager.cc:359] Metadata directory not provided
I20260806 03:47:15.870602 22608 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:47:15.872124 22608 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260806 03:47:15.873982 22637 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:47:15.874166 22608 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20260806 03:47:15.874270 22608 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988004637995-12279-0/minicluster-data/ts-2/data,/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988004637995-12279-0/minicluster-data/ts-2/wal
uuid: "05f018b6c7cd4ee3a66404e6d7ef0729"
format_stamp: "Formatted at 2026-08-06 03:47:14 on dist-test-slave-6ntk"
server_key: "5aeefb93b1f28cd79d07baef53cf30dd"
server_key_iv: "5cbf1f033813c7a21e98a05e8bc060b2"
server_key_version: "encryptionkey@0"
I20260806 03:47:15.874634 22608 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988004637995-12279-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988004637995-12279-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988004637995-12279-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:47:15.885143 22608 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:47:15.887161 22608 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:47:15.887317 22608 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:47:15.887579 22608 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 03:47:15.887903 22608 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 03:47:15.887957 22608 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 03:47:15.888000 22608 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 03:47:15.888028 22608 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 03:47:15.894246 22608 rpc_server.cc:307] RPC server started. Bound to: 127.11.253.195:33553
I20260806 03:47:15.894294 22802 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.253.195:33553 every 8 connection(s)
I20260806 03:47:15.894650 22608 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988004637995-12279-0/minicluster-data/ts-2/data/info.pb
I20260806 03:47:15.895622 12279 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskToVWRe/build/release/bin/kudu as pid 22608
Aug 06 03:47:15 dist-test-slave-6ntk krb5kdc[20936](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988035, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.253.195@KRBTEST.COM for oryx/127.11.253.253@KRBTEST.COM
Aug 06 03:47:15 dist-test-slave-6ntk krb5kdc[20936](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988035, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.253.195@KRBTEST.COM for oryx/127.11.253.254@KRBTEST.COM
Aug 06 03:47:15 dist-test-slave-6ntk krb5kdc[20936](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988035, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.253.195@KRBTEST.COM for oryx/127.11.253.252@KRBTEST.COM
I20260806 03:47:15.909024 21945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:15.895437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.253.195:33867 (local address 127.11.253.253:46317)
0806 03:47:15.895493 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0806 03:47:15.895496 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 03:47:15.895998 (+ 502us) server_negotiation.cc:407] Connection header received
0806 03:47:15.896780 (+ 782us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:47:15.896782 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:47:15.896794 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:47:15.896859 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:47:15.897951 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:15.898707 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:15.900265 (+ 1558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:15.900560 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:15.900603 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:15.904889 (+ 4286us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:47:15.904891 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:47:15.904891 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:47:15.904900 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:47:15.906828 (+ 1928us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:47:15.908038 (+ 1210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:47:15.908039 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:47:15.908039 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:47:15.908091 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:47:15.908298 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:47:15.908299 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:47:15.908299 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:47:15.908415 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:47:15.908458 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0806 03:47:15.908828 (+ 370us) server_negotiation.cc:299] Negotiation successful
0806 03:47:15.908931 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260806 03:47:15.909430 22209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:15.895869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.253.195:36181 (local address 127.11.253.252:40283)
0806 03:47:15.895919 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0806 03:47:15.895922 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 03:47:15.896780 (+ 858us) server_negotiation.cc:407] Connection header received
0806 03:47:15.896891 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:47:15.896892 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:47:15.896906 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:47:15.896939 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:47:15.897938 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:15.898734 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:15.900259 (+ 1525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:15.900561 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:15.900602 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:15.905580 (+ 4978us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:47:15.905581 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:47:15.905581 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:47:15.905588 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:47:15.907541 (+ 1953us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:47:15.908498 (+ 957us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:47:15.908499 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:47:15.908499 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:47:15.908564 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:47:15.908776 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:47:15.908777 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:47:15.908778 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:47:15.908929 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:47:15.908975 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0806 03:47:15.909210 (+ 235us) server_negotiation.cc:299] Negotiation successful
0806 03:47:15.909333 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260806 03:47:15.909516 22812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:15.895432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.253:46317 (local address 127.11.253.195:33867)
0806 03:47:15.895826 (+ 394us) negotiation.cc:107] Waiting for socket to connect
0806 03:47:15.895881 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0806 03:47:15.896531 (+ 650us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:47:15.896996 (+ 465us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:47:15.897001 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:47:15.897497 (+ 496us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:47:15.897799 (+ 302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:15.897805 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:15.898862 (+ 1057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:15.898862 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:15.900176 (+ 1314us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:15.900177 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:15.900687 (+ 510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:15.900688 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:15.900822 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:15.901561 (+ 739us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:47:15.901569 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:47:15.904767 (+ 3198us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:47:15.907657 (+ 2890us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:47:15.907658 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:47:15.907658 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:47:15.907949 (+ 291us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:47:15.908164 (+ 215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:47:15.908164 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:47:15.908165 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:47:15.908223 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:47:15.908506 (+ 283us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:47:15.908506 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:47:15.908666 (+ 160us) client_negotiation.cc:769] Sending connection context
0806 03:47:15.909222 (+ 556us) client_negotiation.cc:240] Negotiation successful
0806 03:47:15.909320 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":352,"mutex_wait_us":66,"spinlock_wait_cycles":384,"thread_start_us":92,"threads_started":1}
I20260806 03:47:15.909548 22814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:15.895725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.252:40283 (local address 127.11.253.195:36181)
0806 03:47:15.896719 (+ 994us) negotiation.cc:107] Waiting for socket to connect
0806 03:47:15.896728 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 03:47:15.896809 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:47:15.897015 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:47:15.897016 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:47:15.897694 (+ 678us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:47:15.897800 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:15.897808 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:15.898867 (+ 1059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:15.898868 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:15.900159 (+ 1291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:15.900161 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:15.900685 (+ 524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:15.900686 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:15.900821 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:15.901990 (+ 1169us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:47:15.901995 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:47:15.905497 (+ 3502us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:47:15.907638 (+ 2141us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:47:15.907639 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:47:15.907640 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:47:15.908401 (+ 761us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:47:15.908643 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:47:15.908643 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:47:15.908644 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:47:15.908700 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:47:15.909036 (+ 336us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:47:15.909037 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:47:15.909080 (+ 43us) client_negotiation.cc:769] Sending connection context
0806 03:47:15.909321 (+ 241us) client_negotiation.cc:240] Negotiation successful
0806 03:47:15.909423 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":976,"mutex_wait_us":106,"thread_start_us":87,"threads_started":1}
I20260806 03:47:15.910054 22806 heartbeater.cc:347] Connected to a master server at 127.11.253.253:46317
I20260806 03:47:15.910146 22803 heartbeater.cc:347] Connected to a master server at 127.11.253.252:40283
I20260806 03:47:15.910162 22806 heartbeater.cc:464] Registering TS with master...
I20260806 03:47:15.910181 22803 heartbeater.cc:464] Registering TS with master...
I20260806 03:47:15.910211 21061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:15.895682 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.253.195:35067 (local address 127.11.253.254:32833)
0806 03:47:15.895736 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0806 03:47:15.895739 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 03:47:15.895970 (+ 231us) server_negotiation.cc:407] Connection header received
0806 03:47:15.896634 (+ 664us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:47:15.896636 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:47:15.896650 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:47:15.896712 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:47:15.897934 (+ 1222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:15.898713 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:15.900258 (+ 1545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:15.900562 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:15.900617 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:15.904274 (+ 3657us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:47:15.904276 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:47:15.904276 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:47:15.904283 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:47:15.907314 (+ 3031us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:47:15.907895 (+ 581us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:47:15.907896 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:47:15.907897 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:47:15.907958 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:47:15.908245 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:47:15.908246 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:47:15.908246 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:47:15.908393 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:47:15.908458 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0806 03:47:15.909441 (+ 983us) server_negotiation.cc:299] Negotiation successful
0806 03:47:15.910099 (+ 658us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260806 03:47:15.910372 22803 heartbeater.cc:511] Master 127.11.253.252:40283 requested a full tablet report, sending...
I20260806 03:47:15.910372 22806 heartbeater.cc:511] Master 127.11.253.253:46317 requested a full tablet report, sending...
I20260806 03:47:15.910843 22148 ts_manager.cc:194] Registered new tserver with Master: 05f018b6c7cd4ee3a66404e6d7ef0729 (127.11.253.195:33553)
I20260806 03:47:15.910843 21871 ts_manager.cc:194] Registered new tserver with Master: 05f018b6c7cd4ee3a66404e6d7ef0729 (127.11.253.195:33553)
I20260806 03:47:15.911881 22813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:15.895432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:32833 (local address 127.11.253.195:35067)
0806 03:47:15.895868 (+ 436us) negotiation.cc:107] Waiting for socket to connect
0806 03:47:15.895881 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 03:47:15.896536 (+ 655us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:47:15.897444 (+ 908us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:47:15.897445 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:47:15.897731 (+ 286us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:47:15.897799 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:15.897806 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:15.898854 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:15.898855 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:15.900160 (+ 1305us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:15.900162 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:15.900678 (+ 516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:15.900679 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:15.900822 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:15.901625 (+ 803us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:47:15.901629 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:47:15.904168 (+ 2539us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:47:15.907428 (+ 3260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:47:15.907438 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:47:15.907441 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:47:15.907805 (+ 364us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:47:15.908027 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:47:15.908028 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:47:15.908028 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:47:15.908174 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:47:15.908486 (+ 312us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:47:15.908496 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:47:15.908666 (+ 170us) client_negotiation.cc:769] Sending connection context
0806 03:47:15.908783 (+ 117us) client_negotiation.cc:240] Negotiation successful
0806 03:47:15.908956 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":414,"mutex_wait_us":126,"thread_start_us":89,"threads_started":1}
I20260806 03:47:15.912331 22804 heartbeater.cc:347] Connected to a master server at 127.11.253.254:32833
I20260806 03:47:15.912396 22804 heartbeater.cc:464] Registering TS with master...
I20260806 03:47:15.912422 22830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:15.897295 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44902 (local address 127.11.253.252:40283)
0806 03:47:15.901697 (+ 4402us) server_negotiation.cc:206] Beginning negotiation
0806 03:47:15.901699 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:47:15.901706 (+ 7us) server_negotiation.cc:407] Connection header received
0806 03:47:15.901725 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:47:15.901726 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:47:15.901736 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:47:15.902868 (+ 1132us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:47:15.903466 (+ 598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:15.904227 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:15.905906 (+ 1679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:15.906186 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:15.906247 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:15.908613 (+ 2366us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:47:15.908614 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:47:15.908614 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:47:15.908621 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:47:15.910769 (+ 2148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:47:15.911232 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:47:15.911234 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:47:15.911234 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:47:15.911313 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:47:15.911558 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:47:15.911559 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:47:15.911559 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:47:15.911676 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:47:15.911849 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0806 03:47:15.912137 (+ 288us) server_negotiation.cc:299] Negotiation successful
0806 03:47:15.912339 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4357,"thread_start_us":127,"threads_started":1}
I20260806 03:47:15.912477 22804 heartbeater.cc:511] Master 127.11.253.254:32833 requested a full tablet report, sending...
I20260806 03:47:15.912842 21005 ts_manager.cc:194] Re-registered known tserver with Master: 05f018b6c7cd4ee3a66404e6d7ef0729 (127.11.253.195:33553)
I20260806 03:47:15.913353 21005 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.11.253.195@KRBTEST.COM'} at 127.11.253.195:35067
I20260806 03:47:15.914374 22829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:15.897310 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42216 (local address 127.11.253.254:32833)
0806 03:47:15.897800 (+ 490us) server_negotiation.cc:206] Beginning negotiation
0806 03:47:15.897803 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 03:47:15.897988 (+ 185us) server_negotiation.cc:407] Connection header received
0806 03:47:15.898011 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:47:15.898012 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:47:15.898024 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:47:15.898080 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:47:15.898870 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:15.899650 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:15.900674 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:15.900961 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:15.901030 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:15.902812 (+ 1782us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:47:15.902813 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:47:15.902814 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:47:15.902822 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:47:15.911745 (+ 8923us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:47:15.912594 (+ 849us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:47:15.912595 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:47:15.912595 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:47:15.912659 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:47:15.913740 (+ 1081us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:47:15.913741 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:47:15.913741 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:47:15.913858 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:47:15.913996 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0806 03:47:15.914092 (+ 96us) server_negotiation.cc:299] Negotiation successful
0806 03:47:15.914181 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":468,"thread_start_us":77,"threads_started":1}
I20260806 03:47:15.915308 22826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:15.897048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39950 (local address 127.11.253.253:46317)
0806 03:47:15.901516 (+ 4468us) server_negotiation.cc:206] Beginning negotiation
0806 03:47:15.901519 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 03:47:15.901526 (+ 7us) server_negotiation.cc:407] Connection header received
0806 03:47:15.901546 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:47:15.901546 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:47:15.901557 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:47:15.903474 (+ 1917us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:47:15.903509 (+ 35us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:15.904227 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:15.905428 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:15.905728 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:15.905829 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:15.907882 (+ 2053us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:47:15.907883 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:47:15.907883 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:47:15.907890 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:47:15.913397 (+ 5507us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:47:15.914388 (+ 991us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:47:15.914389 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:47:15.914389 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:47:15.914457 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:47:15.914691 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:47:15.914692 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:47:15.914692 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:47:15.914828 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:47:15.914901 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0806 03:47:15.915051 (+ 150us) server_negotiation.cc:299] Negotiation successful
0806 03:47:15.915148 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4447,"thread_start_us":76,"threads_started":1}
I20260806 03:47:15.916114 21005 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42216:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 03:47:15.917601 21005 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 03:47:15.925652 22843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:15.921431 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44750 (local address 127.11.253.193:34025)
0806 03:47:15.921614 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0806 03:47:15.921620 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 03:47:15.921632 (+ 12us) server_negotiation.cc:407] Connection header received
0806 03:47:15.921675 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:47:15.921677 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:47:15.921696 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:47:15.921976 (+ 280us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 03:47:15.922049 (+ 73us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:15.923152 (+ 1103us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:15.924608 (+ 1456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:15.925291 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:15.925388 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:15.925447 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0806 03:47:15.925519 (+ 72us) server_negotiation.cc:299] Negotiation successful
0806 03:47:15.925562 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":101,"threads_started":1}
I20260806 03:47:15.925659 22841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:15.921226 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.193:34025 (local address 127.0.0.1:44750)
0806 03:47:15.921387 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0806 03:47:15.921400 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 03:47:15.921456 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:47:15.921803 (+ 347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:47:15.921804 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:47:15.921806 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 03:47:15.921887 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:15.921890 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:15.923242 (+ 1352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:15.923243 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:15.924513 (+ 1270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:15.924516 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:15.925382 (+ 866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:15.925383 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:15.925448 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:15.925452 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 03:47:15.925508 (+ 56us) client_negotiation.cc:240] Negotiation successful
0806 03:47:15.925554 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":119,"threads_started":1}
I20260806 03:47:15.926067 22842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:15.921287 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.195:33553 (local address 127.0.0.1:45240)
0806 03:47:15.921451 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0806 03:47:15.921462 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 03:47:15.921735 (+ 273us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:47:15.922367 (+ 632us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:47:15.922367 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:47:15.922370 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 03:47:15.922407 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:15.922409 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:15.923694 (+ 1285us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:15.923695 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:15.924641 (+ 946us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:15.924644 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:15.925539 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:15.925539 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:15.925589 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:15.925592 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 03:47:15.925638 (+ 46us) client_negotiation.cc:240] Negotiation successful
0806 03:47:15.925671 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"mutex_wait_us":37,"thread_start_us":116,"threads_started":1}
I20260806 03:47:15.926378 22847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:15.921536 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45240 (local address 127.11.253.195:33553)
0806 03:47:15.922189 (+ 653us) server_negotiation.cc:206] Beginning negotiation
0806 03:47:15.922194 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 03:47:15.922203 (+ 9us) server_negotiation.cc:407] Connection header received
0806 03:47:15.922239 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:47:15.922240 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:47:15.922256 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:47:15.922341 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 03:47:15.922489 (+ 148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:15.923593 (+ 1104us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:15.924729 (+ 1136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:15.925330 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:15.925440 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:15.925505 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0806 03:47:15.925724 (+ 219us) server_negotiation.cc:299] Negotiation successful
0806 03:47:15.926281 (+ 557us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":617,"thread_start_us":96,"threads_started":1}
I20260806 03:47:15.926406 22853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:15.921895 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38472 (local address 127.11.253.194:45307)
0806 03:47:15.922528 (+ 633us) server_negotiation.cc:206] Beginning negotiation
0806 03:47:15.922532 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 03:47:15.922541 (+ 9us) server_negotiation.cc:407] Connection header received
0806 03:47:15.922576 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:47:15.922578 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:47:15.922593 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:47:15.922815 (+ 222us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 03:47:15.922875 (+ 60us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:15.923714 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:15.924993 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:15.925695 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:15.926128 (+ 433us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:15.926215 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0806 03:47:15.926244 (+ 29us) server_negotiation.cc:299] Negotiation successful
0806 03:47:15.926308 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":593,"thread_start_us":164,"threads_started":1}
I20260806 03:47:15.926514 22849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:15.921792 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.194:45307 (local address 127.0.0.1:38472)
0806 03:47:15.922005 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0806 03:47:15.922014 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 03:47:15.922060 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:47:15.922683 (+ 623us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:47:15.922684 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:47:15.922687 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 03:47:15.922730 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:15.922732 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:15.923799 (+ 1067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:15.923801 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:15.924900 (+ 1099us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:15.924903 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:15.925806 (+ 903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:15.925807 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:15.925942 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:15.925947 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 03:47:15.925983 (+ 36us) client_negotiation.cc:240] Negotiation successful
0806 03:47:15.926037 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":104,"threads_started":1}
I20260806 03:47:15.926785 22311 tablet_service.cc:1467] Processing CreateTablet for tablet 31bd1739b18f4080b8ca60699c3a43f6 (DEFAULT_TABLE table=test-table [id=a184c3606765450a91ec79fa88bc01fd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 03:47:15.926918 22702 tablet_service.cc:1467] Processing CreateTablet for tablet 31bd1739b18f4080b8ca60699c3a43f6 (DEFAULT_TABLE table=test-table [id=a184c3606765450a91ec79fa88bc01fd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 03:47:15.927121 22311 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 31bd1739b18f4080b8ca60699c3a43f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:47:15.927220 22702 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 31bd1739b18f4080b8ca60699c3a43f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:47:15.929906 22855 tablet_bootstrap.cc:492] T 31bd1739b18f4080b8ca60699c3a43f6 P 21d27ac5b0aa4681b44bc0482b668d54: Bootstrap starting.
I20260806 03:47:15.930275 22854 tablet_bootstrap.cc:492] T 31bd1739b18f4080b8ca60699c3a43f6 P 05f018b6c7cd4ee3a66404e6d7ef0729: Bootstrap starting.
I20260806 03:47:15.930986 22854 tablet_bootstrap.cc:654] T 31bd1739b18f4080b8ca60699c3a43f6 P 05f018b6c7cd4ee3a66404e6d7ef0729: Neither blocks nor log segments found. Creating new log.
I20260806 03:47:15.931021 22504 tablet_service.cc:1467] Processing CreateTablet for tablet 31bd1739b18f4080b8ca60699c3a43f6 (DEFAULT_TABLE table=test-table [id=a184c3606765450a91ec79fa88bc01fd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 03:47:15.931339 22854 log.cc:824] T 31bd1739b18f4080b8ca60699c3a43f6 P 05f018b6c7cd4ee3a66404e6d7ef0729: Log is configured to *not* fsync() on all Append() calls
I20260806 03:47:15.931427 22504 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 31bd1739b18f4080b8ca60699c3a43f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:47:15.932138 22854 tablet_bootstrap.cc:492] T 31bd1739b18f4080b8ca60699c3a43f6 P 05f018b6c7cd4ee3a66404e6d7ef0729: No bootstrap required, opened a new log
I20260806 03:47:15.932175 22855 tablet_bootstrap.cc:654] T 31bd1739b18f4080b8ca60699c3a43f6 P 21d27ac5b0aa4681b44bc0482b668d54: Neither blocks nor log segments found. Creating new log.
I20260806 03:47:15.932222 22854 ts_tablet_manager.cc:1397] T 31bd1739b18f4080b8ca60699c3a43f6 P 05f018b6c7cd4ee3a66404e6d7ef0729: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 03:47:15.932515 22855 log.cc:824] T 31bd1739b18f4080b8ca60699c3a43f6 P 21d27ac5b0aa4681b44bc0482b668d54: Log is configured to *not* fsync() on all Append() calls
I20260806 03:47:15.933526 22855 tablet_bootstrap.cc:492] T 31bd1739b18f4080b8ca60699c3a43f6 P 21d27ac5b0aa4681b44bc0482b668d54: No bootstrap required, opened a new log
I20260806 03:47:15.933704 22855 ts_tablet_manager.cc:1397] T 31bd1739b18f4080b8ca60699c3a43f6 P 21d27ac5b0aa4681b44bc0482b668d54: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260806 03:47:15.934309 22854 raft_consensus.cc:348] T 31bd1739b18f4080b8ca60699c3a43f6 P 05f018b6c7cd4ee3a66404e6d7ef0729 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21d27ac5b0aa4681b44bc0482b668d54" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 34025 } } peers { permanent_uuid: "05f018b6c7cd4ee3a66404e6d7ef0729" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 33553 } } peers { permanent_uuid: "0fa8b770e150486aae31b4df6ad38a6e" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 45307 } }
I20260806 03:47:15.934464 22854 raft_consensus.cc:374] T 31bd1739b18f4080b8ca60699c3a43f6 P 05f018b6c7cd4ee3a66404e6d7ef0729 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:47:15.934508 22854 raft_consensus.cc:729] T 31bd1739b18f4080b8ca60699c3a43f6 P 05f018b6c7cd4ee3a66404e6d7ef0729 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 05f018b6c7cd4ee3a66404e6d7ef0729, State: Initialized, Role: FOLLOWER
I20260806 03:47:15.934628 22854 consensus_queue.cc:260] T 31bd1739b18f4080b8ca60699c3a43f6 P 05f018b6c7cd4ee3a66404e6d7ef0729 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21d27ac5b0aa4681b44bc0482b668d54" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 34025 } } peers { permanent_uuid: "05f018b6c7cd4ee3a66404e6d7ef0729" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 33553 } } peers { permanent_uuid: "0fa8b770e150486aae31b4df6ad38a6e" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 45307 } }
I20260806 03:47:15.934930 22854 ts_tablet_manager.cc:1428] T 31bd1739b18f4080b8ca60699c3a43f6 P 05f018b6c7cd4ee3a66404e6d7ef0729: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 03:47:15.935041 22804 heartbeater.cc:503] Master 127.11.253.254:32833 was elected leader, sending a full tablet report...
I20260806 03:47:15.935189 22860 tablet_bootstrap.cc:492] T 31bd1739b18f4080b8ca60699c3a43f6 P 0fa8b770e150486aae31b4df6ad38a6e: Bootstrap starting.
I20260806 03:47:15.935874 22860 tablet_bootstrap.cc:654] T 31bd1739b18f4080b8ca60699c3a43f6 P 0fa8b770e150486aae31b4df6ad38a6e: Neither blocks nor log segments found. Creating new log.
I20260806 03:47:15.936235 22860 log.cc:824] T 31bd1739b18f4080b8ca60699c3a43f6 P 0fa8b770e150486aae31b4df6ad38a6e: Log is configured to *not* fsync() on all Append() calls
I20260806 03:47:15.937126 22860 tablet_bootstrap.cc:492] T 31bd1739b18f4080b8ca60699c3a43f6 P 0fa8b770e150486aae31b4df6ad38a6e: No bootstrap required, opened a new log
I20260806 03:47:15.937214 22860 ts_tablet_manager.cc:1397] T 31bd1739b18f4080b8ca60699c3a43f6 P 0fa8b770e150486aae31b4df6ad38a6e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 03:47:15.937371 22855 raft_consensus.cc:348] T 31bd1739b18f4080b8ca60699c3a43f6 P 21d27ac5b0aa4681b44bc0482b668d54 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21d27ac5b0aa4681b44bc0482b668d54" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 34025 } } peers { permanent_uuid: "05f018b6c7cd4ee3a66404e6d7ef0729" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 33553 } } peers { permanent_uuid: "0fa8b770e150486aae31b4df6ad38a6e" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 45307 } }
I20260806 03:47:15.937494 22855 raft_consensus.cc:374] T 31bd1739b18f4080b8ca60699c3a43f6 P 21d27ac5b0aa4681b44bc0482b668d54 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:47:15.937516 22855 raft_consensus.cc:729] T 31bd1739b18f4080b8ca60699c3a43f6 P 21d27ac5b0aa4681b44bc0482b668d54 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 21d27ac5b0aa4681b44bc0482b668d54, State: Initialized, Role: FOLLOWER
I20260806 03:47:15.937664 22855 consensus_queue.cc:260] T 31bd1739b18f4080b8ca60699c3a43f6 P 21d27ac5b0aa4681b44bc0482b668d54 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21d27ac5b0aa4681b44bc0482b668d54" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 34025 } } peers { permanent_uuid: "05f018b6c7cd4ee3a66404e6d7ef0729" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 33553 } } peers { permanent_uuid: "0fa8b770e150486aae31b4df6ad38a6e" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 45307 } }
I20260806 03:47:15.938078 22855 ts_tablet_manager.cc:1428] T 31bd1739b18f4080b8ca60699c3a43f6 P 21d27ac5b0aa4681b44bc0482b668d54: Time spent starting tablet: real 0.004s user 0.002s sys 0.001s
I20260806 03:47:15.938279 22399 heartbeater.cc:503] Master 127.11.253.254:32833 was elected leader, sending a full tablet report...
I20260806 03:47:15.939585 22860 raft_consensus.cc:348] T 31bd1739b18f4080b8ca60699c3a43f6 P 0fa8b770e150486aae31b4df6ad38a6e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21d27ac5b0aa4681b44bc0482b668d54" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 34025 } } peers { permanent_uuid: "05f018b6c7cd4ee3a66404e6d7ef0729" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 33553 } } peers { permanent_uuid: "0fa8b770e150486aae31b4df6ad38a6e" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 45307 } }
I20260806 03:47:15.939705 22860 raft_consensus.cc:374] T 31bd1739b18f4080b8ca60699c3a43f6 P 0fa8b770e150486aae31b4df6ad38a6e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:47:15.939728 22860 raft_consensus.cc:729] T 31bd1739b18f4080b8ca60699c3a43f6 P 0fa8b770e150486aae31b4df6ad38a6e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0fa8b770e150486aae31b4df6ad38a6e, State: Initialized, Role: FOLLOWER
I20260806 03:47:15.939801 22860 consensus_queue.cc:260] T 31bd1739b18f4080b8ca60699c3a43f6 P 0fa8b770e150486aae31b4df6ad38a6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21d27ac5b0aa4681b44bc0482b668d54" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 34025 } } peers { permanent_uuid: "05f018b6c7cd4ee3a66404e6d7ef0729" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 33553 } } peers { permanent_uuid: "0fa8b770e150486aae31b4df6ad38a6e" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 45307 } }
I20260806 03:47:15.940090 22860 ts_tablet_manager.cc:1428] T 31bd1739b18f4080b8ca60699c3a43f6 P 0fa8b770e150486aae31b4df6ad38a6e: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 03:47:15.940186 22602 heartbeater.cc:503] Master 127.11.253.254:32833 was elected leader, sending a full tablet report...
I20260806 03:47:15.948866 22842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:15.946483 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.252:40283 (local address 127.0.0.1:44916)
0806 03:47:15.946545 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0806 03:47:15.946555 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 03:47:15.946595 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:47:15.946729 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:47:15.946729 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:47:15.946730 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 03:47:15.946775 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:15.946777 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:15.947414 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:15.947414 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:15.948124 (+ 710us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:15.948126 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:15.948674 (+ 548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:15.948675 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:15.948719 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:15.948722 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 03:47:15.948763 (+ 41us) client_negotiation.cc:240] Negotiation successful
0806 03:47:15.948803 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260806 03:47:15.948871 22830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:15.946538 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44916 (local address 127.11.253.252:40283)
0806 03:47:15.946594 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0806 03:47:15.946596 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:47:15.946612 (+ 16us) server_negotiation.cc:407] Connection header received
0806 03:47:15.946636 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:47:15.946637 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:47:15.946657 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:47:15.946720 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 03:47:15.946831 (+ 111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:15.947348 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:15.948200 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:15.948605 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:15.948667 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:15.948715 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0806 03:47:15.948788 (+ 73us) server_negotiation.cc:299] Negotiation successful
0806 03:47:15.948819 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260806 03:47:15.949290 22158 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 492ed5d42bfe4567aa936d6a80e98096 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.9] Dedup: 1.7->[1.8-1.9]
I20260806 03:47:15.950280 22868 sys_catalog.cc:455] T 00000000000000000000000000000000 P 492ed5d42bfe4567aa936d6a80e98096 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 46d7cfc9ed574f1794039e138d8f6220. Latest consensus state: current_term: 1 leader_uuid: "46d7cfc9ed574f1794039e138d8f6220" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "46d7cfc9ed574f1794039e138d8f6220" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 32833 } } peers { permanent_uuid: "85510334aee348ffbc06964d94b3087b" member_type: VOTER last_known_addr { host: "127.11.253.253" port: 46317 } attrs { promote: false } } peers { permanent_uuid: "492ed5d42bfe4567aa936d6a80e98096" member_type: NON_VOTER last_known_addr { host: "127.11.253.252" port: 40283 } attrs { promote: true } } }
I20260806 03:47:15.950379 22868 sys_catalog.cc:458] T 00000000000000000000000000000000 P 492ed5d42bfe4567aa936d6a80e98096 [sys.catalog]: This master's current role is: LEARNER
W20260806 03:47:16.001803 22604 tablet.cc:2369] T 31bd1739b18f4080b8ca60699c3a43f6 P 0fa8b770e150486aae31b4df6ad38a6e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 03:47:16.107167 22404 tablet.cc:2369] T 31bd1739b18f4080b8ca60699c3a43f6 P 21d27ac5b0aa4681b44bc0482b668d54: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 03:47:16.142884 22859 raft_consensus.cc:482] T 31bd1739b18f4080b8ca60699c3a43f6 P 05f018b6c7cd4ee3a66404e6d7ef0729 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 03:47:16.143071 22859 raft_consensus.cc:504] T 31bd1739b18f4080b8ca60699c3a43f6 P 05f018b6c7cd4ee3a66404e6d7ef0729 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21d27ac5b0aa4681b44bc0482b668d54" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 34025 } } peers { permanent_uuid: "05f018b6c7cd4ee3a66404e6d7ef0729" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 33553 } } peers { permanent_uuid: "0fa8b770e150486aae31b4df6ad38a6e" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 45307 } }
I20260806 03:47:16.143539 22859 leader_election.cc:288] T 31bd1739b18f4080b8ca60699c3a43f6 P 05f018b6c7cd4ee3a66404e6d7ef0729 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 21d27ac5b0aa4681b44bc0482b668d54 (127.11.253.193:34025), 0fa8b770e150486aae31b4df6ad38a6e (127.11.253.194:45307)
W20260806 03:47:16.145432 22808 tablet.cc:2369] T 31bd1739b18f4080b8ca60699c3a43f6 P 05f018b6c7cd4ee3a66404e6d7ef0729: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 03:47:16.146553 22853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:16.143840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.253.195:51323 (local address 127.11.253.194:45307)
0806 03:47:16.143887 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0806 03:47:16.143889 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:47:16.143900 (+ 11us) server_negotiation.cc:407] Connection header received
0806 03:47:16.143964 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:47:16.143965 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:47:16.143980 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:47:16.144043 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 03:47:16.144212 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:16.144691 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:16.145763 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:16.146200 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:16.146279 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:16.146288 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0806 03:47:16.146439 (+ 151us) server_negotiation.cc:299] Negotiation successful
0806 03:47:16.146476 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260806 03:47:16.146574 22843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:16.143805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.253.195:39451 (local address 127.11.253.193:34025)
0806 03:47:16.143858 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0806 03:47:16.143861 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 03:47:16.143903 (+ 42us) server_negotiation.cc:407] Connection header received
0806 03:47:16.143984 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:47:16.143985 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:47:16.143998 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:47:16.144062 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 03:47:16.144233 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:16.144698 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:16.145757 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:16.146203 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:16.146300 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:16.146309 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0806 03:47:16.146464 (+ 155us) server_negotiation.cc:299] Negotiation successful
0806 03:47:16.146501 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260806 03:47:16.146584 22814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:16.143751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.193:34025 (local address 127.11.253.195:39451)
0806 03:47:16.143855 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0806 03:47:16.143867 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 03:47:16.143919 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:47:16.144065 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:47:16.144066 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:47:16.144067 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 03:47:16.144149 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:16.144150 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:16.144777 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:16.144778 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:16.145624 (+ 846us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:16.145627 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:16.146302 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:16.146303 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:16.146377 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:16.146381 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 03:47:16.146456 (+ 75us) client_negotiation.cc:240] Negotiation successful
0806 03:47:16.146496 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":89,"mutex_wait_us":88}
I20260806 03:47:16.146589 22813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:16.143739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.194:45307 (local address 127.11.253.195:51323)
0806 03:47:16.143807 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0806 03:47:16.143821 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 03:47:16.143893 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:47:16.144046 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:47:16.144047 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:47:16.144049 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 03:47:16.144134 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:16.144136 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:16.144773 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:16.144774 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:16.145667 (+ 893us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:16.145670 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:16.146279 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:16.146280 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:16.146366 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:16.146369 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 03:47:16.146434 (+ 65us) client_negotiation.cc:240] Negotiation successful
0806 03:47:16.146476 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43}
I20260806 03:47:16.146945 22339 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31bd1739b18f4080b8ca60699c3a43f6" candidate_uuid: "05f018b6c7cd4ee3a66404e6d7ef0729" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "21d27ac5b0aa4681b44bc0482b668d54" is_pre_election: true
I20260806 03:47:16.146942 22531 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31bd1739b18f4080b8ca60699c3a43f6" candidate_uuid: "05f018b6c7cd4ee3a66404e6d7ef0729" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0fa8b770e150486aae31b4df6ad38a6e" is_pre_election: true
I20260806 03:47:16.147099 22531 raft_consensus.cc:2454] T 31bd1739b18f4080b8ca60699c3a43f6 P 0fa8b770e150486aae31b4df6ad38a6e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05f018b6c7cd4ee3a66404e6d7ef0729 in term 0.
I20260806 03:47:16.147109 22339 raft_consensus.cc:2454] T 31bd1739b18f4080b8ca60699c3a43f6 P 21d27ac5b0aa4681b44bc0482b668d54 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05f018b6c7cd4ee3a66404e6d7ef0729 in term 0.
I20260806 03:47:16.147320 22639 leader_election.cc:302] T 31bd1739b18f4080b8ca60699c3a43f6 P 05f018b6c7cd4ee3a66404e6d7ef0729 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05f018b6c7cd4ee3a66404e6d7ef0729, 0fa8b770e150486aae31b4df6ad38a6e; no voters:
I20260806 03:47:16.147468 22859 raft_consensus.cc:2781] T 31bd1739b18f4080b8ca60699c3a43f6 P 05f018b6c7cd4ee3a66404e6d7ef0729 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 03:47:16.147535 22859 raft_consensus.cc:482] T 31bd1739b18f4080b8ca60699c3a43f6 P 05f018b6c7cd4ee3a66404e6d7ef0729 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 03:47:16.147591 22859 raft_consensus.cc:3037] T 31bd1739b18f4080b8ca60699c3a43f6 P 05f018b6c7cd4ee3a66404e6d7ef0729 [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:47:16.148399 22859 raft_consensus.cc:504] T 31bd1739b18f4080b8ca60699c3a43f6 P 05f018b6c7cd4ee3a66404e6d7ef0729 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21d27ac5b0aa4681b44bc0482b668d54" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 34025 } } peers { permanent_uuid: "05f018b6c7cd4ee3a66404e6d7ef0729" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 33553 } } peers { permanent_uuid: "0fa8b770e150486aae31b4df6ad38a6e" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 45307 } }
I20260806 03:47:16.148535 22859 leader_election.cc:288] T 31bd1739b18f4080b8ca60699c3a43f6 P 05f018b6c7cd4ee3a66404e6d7ef0729 [CANDIDATE]: Term 1 election: Requested vote from peers 21d27ac5b0aa4681b44bc0482b668d54 (127.11.253.193:34025), 0fa8b770e150486aae31b4df6ad38a6e (127.11.253.194:45307)
I20260806 03:47:16.148672 22339 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31bd1739b18f4080b8ca60699c3a43f6" candidate_uuid: "05f018b6c7cd4ee3a66404e6d7ef0729" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "21d27ac5b0aa4681b44bc0482b668d54"
I20260806 03:47:16.148741 22531 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31bd1739b18f4080b8ca60699c3a43f6" candidate_uuid: "05f018b6c7cd4ee3a66404e6d7ef0729" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0fa8b770e150486aae31b4df6ad38a6e"
I20260806 03:47:16.148759 22339 raft_consensus.cc:3037] T 31bd1739b18f4080b8ca60699c3a43f6 P 21d27ac5b0aa4681b44bc0482b668d54 [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:47:16.148820 22531 raft_consensus.cc:3037] T 31bd1739b18f4080b8ca60699c3a43f6 P 0fa8b770e150486aae31b4df6ad38a6e [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:47:16.149634 22339 raft_consensus.cc:2454] T 31bd1739b18f4080b8ca60699c3a43f6 P 21d27ac5b0aa4681b44bc0482b668d54 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05f018b6c7cd4ee3a66404e6d7ef0729 in term 1.
I20260806 03:47:16.149770 22531 raft_consensus.cc:2454] T 31bd1739b18f4080b8ca60699c3a43f6 P 0fa8b770e150486aae31b4df6ad38a6e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05f018b6c7cd4ee3a66404e6d7ef0729 in term 1.
I20260806 03:47:16.149832 22643 leader_election.cc:302] T 31bd1739b18f4080b8ca60699c3a43f6 P 05f018b6c7cd4ee3a66404e6d7ef0729 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05f018b6c7cd4ee3a66404e6d7ef0729, 21d27ac5b0aa4681b44bc0482b668d54; no voters:
I20260806 03:47:16.149940 22859 raft_consensus.cc:2781] T 31bd1739b18f4080b8ca60699c3a43f6 P 05f018b6c7cd4ee3a66404e6d7ef0729 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 03:47:16.150110 22859 raft_consensus.cc:686] T 31bd1739b18f4080b8ca60699c3a43f6 P 05f018b6c7cd4ee3a66404e6d7ef0729 [term 1 LEADER]: Becoming Leader. State: Replica: 05f018b6c7cd4ee3a66404e6d7ef0729, State: Running, Role: LEADER
I20260806 03:47:16.150223 22859 consensus_queue.cc:237] T 31bd1739b18f4080b8ca60699c3a43f6 P 05f018b6c7cd4ee3a66404e6d7ef0729 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21d27ac5b0aa4681b44bc0482b668d54" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 34025 } } peers { permanent_uuid: "05f018b6c7cd4ee3a66404e6d7ef0729" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 33553 } } peers { permanent_uuid: "0fa8b770e150486aae31b4df6ad38a6e" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 45307 } }
I20260806 03:47:16.151031 21005 catalog_manager.cc:5495] T 31bd1739b18f4080b8ca60699c3a43f6 P 05f018b6c7cd4ee3a66404e6d7ef0729 reported cstate change: term changed from 0 to 1, leader changed from <none> to 05f018b6c7cd4ee3a66404e6d7ef0729 (127.11.253.195). New cstate: current_term: 1 leader_uuid: "05f018b6c7cd4ee3a66404e6d7ef0729" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21d27ac5b0aa4681b44bc0482b668d54" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 34025 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "05f018b6c7cd4ee3a66404e6d7ef0729" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 33553 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0fa8b770e150486aae31b4df6ad38a6e" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 45307 } health_report { overall_health: UNKNOWN } } }
I20260806 03:47:16.151696 22833 sys_catalog.cc:455] T 00000000000000000000000000000000 P 46d7cfc9ed574f1794039e138d8f6220 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "46d7cfc9ed574f1794039e138d8f6220" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "46d7cfc9ed574f1794039e138d8f6220" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 32833 } } peers { permanent_uuid: "85510334aee348ffbc06964d94b3087b" member_type: VOTER last_known_addr { host: "127.11.253.253" port: 46317 } attrs { promote: false } } peers { permanent_uuid: "492ed5d42bfe4567aa936d6a80e98096" member_type: NON_VOTER last_known_addr { host: "127.11.253.252" port: 40283 } attrs { promote: true } } }
I20260806 03:47:16.151810 22833 sys_catalog.cc:458] T 00000000000000000000000000000000 P 46d7cfc9ed574f1794039e138d8f6220 [sys.catalog]: This master's current role is: LEADER
I20260806 03:47:16.152563 22094 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 46d7cfc9ed574f1794039e138d8f6220: attempting to promote NON_VOTER 492ed5d42bfe4567aa936d6a80e98096 to VOTER
I20260806 03:47:16.152693 22094 consensus_queue.cc:237] T 00000000000000000000000000000000 P 46d7cfc9ed574f1794039e138d8f6220 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 1.10, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "46d7cfc9ed574f1794039e138d8f6220" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 32833 } } peers { permanent_uuid: "85510334aee348ffbc06964d94b3087b" member_type: VOTER last_known_addr { host: "127.11.253.253" port: 46317 } attrs { promote: false } } peers { permanent_uuid: "492ed5d42bfe4567aa936d6a80e98096" member_type: VOTER last_known_addr { host: "127.11.253.252" port: 40283 } attrs { promote: false } }
I20260806 03:47:16.153247 21890 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 85510334aee348ffbc06964d94b3087b [term 1 FOLLOWER]: Refusing update from remote peer 46d7cfc9ed574f1794039e138d8f6220: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20260806 03:47:16.153476 22158 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 492ed5d42bfe4567aa936d6a80e98096 [term 1 LEARNER]: Refusing update from remote peer 46d7cfc9ed574f1794039e138d8f6220: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20260806 03:47:16.153460 22833 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 46d7cfc9ed574f1794039e138d8f6220 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85510334aee348ffbc06964d94b3087b" member_type: VOTER last_known_addr { host: "127.11.253.253" port: 46317 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260806 03:47:16.153659 22833 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 46d7cfc9ed574f1794039e138d8f6220 [LEADER]: Connected to new peer: Peer: permanent_uuid: "492ed5d42bfe4567aa936d6a80e98096" member_type: VOTER last_known_addr { host: "127.11.253.252" port: 40283 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260806 03:47:16.154084 22878 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 46d7cfc9ed574f1794039e138d8f6220 [term 1 LEADER]: Committing config change with OpId 1.11: config changed from index 7 to 11, 492ed5d42bfe4567aa936d6a80e98096 (127.11.253.252) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "46d7cfc9ed574f1794039e138d8f6220" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 32833 } } peers { permanent_uuid: "85510334aee348ffbc06964d94b3087b" member_type: VOTER last_known_addr { host: "127.11.253.253" port: 46317 } attrs { promote: false } } peers { permanent_uuid: "492ed5d42bfe4567aa936d6a80e98096" member_type: VOTER last_known_addr { host: "127.11.253.252" port: 40283 } attrs { promote: false } } }
I20260806 03:47:16.154206 21890 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 85510334aee348ffbc06964d94b3087b [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 7 to 11, 492ed5d42bfe4567aa936d6a80e98096 (127.11.253.252) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "46d7cfc9ed574f1794039e138d8f6220" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 32833 } } peers { permanent_uuid: "85510334aee348ffbc06964d94b3087b" member_type: VOTER last_known_addr { host: "127.11.253.253" port: 46317 } attrs { promote: false } } peers { permanent_uuid: "492ed5d42bfe4567aa936d6a80e98096" member_type: VOTER last_known_addr { host: "127.11.253.252" port: 40283 } attrs { promote: false } } }
I20260806 03:47:16.154440 22158 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 492ed5d42bfe4567aa936d6a80e98096 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 7 to 11, 492ed5d42bfe4567aa936d6a80e98096 (127.11.253.252) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "46d7cfc9ed574f1794039e138d8f6220" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 32833 } } peers { permanent_uuid: "85510334aee348ffbc06964d94b3087b" member_type: VOTER last_known_addr { host: "127.11.253.253" port: 46317 } attrs { promote: false } } peers { permanent_uuid: "492ed5d42bfe4567aa936d6a80e98096" member_type: VOTER last_known_addr { host: "127.11.253.252" port: 40283 } attrs { promote: false } } }
I20260806 03:47:16.155026 22880 sys_catalog.cc:455] T 00000000000000000000000000000000 P 85510334aee348ffbc06964d94b3087b [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "46d7cfc9ed574f1794039e138d8f6220" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "46d7cfc9ed574f1794039e138d8f6220" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 32833 } } peers { permanent_uuid: "85510334aee348ffbc06964d94b3087b" member_type: VOTER last_known_addr { host: "127.11.253.253" port: 46317 } attrs { promote: false } } peers { permanent_uuid: "492ed5d42bfe4567aa936d6a80e98096" member_type: VOTER last_known_addr { host: "127.11.253.252" port: 40283 } attrs { promote: false } } }
I20260806 03:47:16.155118 22880 sys_catalog.cc:458] T 00000000000000000000000000000000 P 85510334aee348ffbc06964d94b3087b [sys.catalog]: This master's current role is: FOLLOWER
I20260806 03:47:16.155243 22833 sys_catalog.cc:455] T 00000000000000000000000000000000 P 46d7cfc9ed574f1794039e138d8f6220 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "46d7cfc9ed574f1794039e138d8f6220" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "46d7cfc9ed574f1794039e138d8f6220" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 32833 } } peers { permanent_uuid: "85510334aee348ffbc06964d94b3087b" member_type: VOTER last_known_addr { host: "127.11.253.253" port: 46317 } attrs { promote: false } } peers { permanent_uuid: "492ed5d42bfe4567aa936d6a80e98096" member_type: VOTER last_known_addr { host: "127.11.253.252" port: 40283 } attrs { promote: false } } }
I20260806 03:47:16.155256 22094 sys_catalog.cc:455] T 00000000000000000000000000000000 P 46d7cfc9ed574f1794039e138d8f6220 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "46d7cfc9ed574f1794039e138d8f6220" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "46d7cfc9ed574f1794039e138d8f6220" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 32833 } } peers { permanent_uuid: "85510334aee348ffbc06964d94b3087b" member_type: VOTER last_known_addr { host: "127.11.253.253" port: 46317 } attrs { promote: false } } peers { permanent_uuid: "492ed5d42bfe4567aa936d6a80e98096" member_type: VOTER last_known_addr { host: "127.11.253.252" port: 40283 } attrs { promote: false } } }
I20260806 03:47:16.155324 22833 sys_catalog.cc:458] T 00000000000000000000000000000000 P 46d7cfc9ed574f1794039e138d8f6220 [sys.catalog]: This master's current role is: LEADER
I20260806 03:47:16.155329 22094 sys_catalog.cc:458] T 00000000000000000000000000000000 P 46d7cfc9ed574f1794039e138d8f6220 [sys.catalog]: This master's current role is: LEADER
I20260806 03:47:16.155427 22878 sys_catalog.cc:455] T 00000000000000000000000000000000 P 46d7cfc9ed574f1794039e138d8f6220 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "46d7cfc9ed574f1794039e138d8f6220" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "46d7cfc9ed574f1794039e138d8f6220" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 32833 } } peers { permanent_uuid: "85510334aee348ffbc06964d94b3087b" member_type: VOTER last_known_addr { host: "127.11.253.253" port: 46317 } attrs { promote: false } } peers { permanent_uuid: "492ed5d42bfe4567aa936d6a80e98096" member_type: VOTER last_known_addr { host: "127.11.253.252" port: 40283 } attrs { promote: false } } }
I20260806 03:47:16.155548 22878 sys_catalog.cc:458] T 00000000000000000000000000000000 P 46d7cfc9ed574f1794039e138d8f6220 [sys.catalog]: This master's current role is: LEADER
I20260806 03:47:16.155566 22868 sys_catalog.cc:455] T 00000000000000000000000000000000 P 492ed5d42bfe4567aa936d6a80e98096 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "46d7cfc9ed574f1794039e138d8f6220" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "46d7cfc9ed574f1794039e138d8f6220" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 32833 } } peers { permanent_uuid: "85510334aee348ffbc06964d94b3087b" member_type: VOTER last_known_addr { host: "127.11.253.253" port: 46317 } attrs { promote: false } } peers { permanent_uuid: "492ed5d42bfe4567aa936d6a80e98096" member_type: VOTER last_known_addr { host: "127.11.253.252" port: 40283 } attrs { promote: false } } }
I20260806 03:47:16.155656 22868 sys_catalog.cc:458] T 00000000000000000000000000000000 P 492ed5d42bfe4567aa936d6a80e98096 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 03:47:16.208737 22847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:16.206053 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45244 (local address 127.11.253.195:33553)
0806 03:47:16.206135 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0806 03:47:16.206138 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 03:47:16.206150 (+ 12us) server_negotiation.cc:407] Connection header received
0806 03:47:16.206205 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:47:16.206206 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:47:16.206227 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:47:16.206318 (+ 91us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 03:47:16.206477 (+ 159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:16.207013 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:16.207847 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:16.208076 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:16.208156 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:16.208370 (+ 214us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 03:47:16.208482 (+ 112us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 03:47:16.208587 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0806 03:47:16.208615 (+ 28us) server_negotiation.cc:299] Negotiation successful
0806 03:47:16.208640 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260806 03:47:16.210850 22531 raft_consensus.cc:1261] T 31bd1739b18f4080b8ca60699c3a43f6 P 0fa8b770e150486aae31b4df6ad38a6e [term 1 FOLLOWER]: Refusing update from remote peer 05f018b6c7cd4ee3a66404e6d7ef0729: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 03:47:16.210873 22339 raft_consensus.cc:1261] T 31bd1739b18f4080b8ca60699c3a43f6 P 21d27ac5b0aa4681b44bc0482b668d54 [term 1 FOLLOWER]: Refusing update from remote peer 05f018b6c7cd4ee3a66404e6d7ef0729: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 03:47:16.211231 22859 consensus_queue.cc:1035] T 31bd1739b18f4080b8ca60699c3a43f6 P 05f018b6c7cd4ee3a66404e6d7ef0729 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0fa8b770e150486aae31b4df6ad38a6e" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 45307 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 03:47:16.211427 22875 consensus_queue.cc:1035] T 31bd1739b18f4080b8ca60699c3a43f6 P 05f018b6c7cd4ee3a66404e6d7ef0729 [LEADER]: Connected to new peer: Peer: permanent_uuid: "21d27ac5b0aa4681b44bc0482b668d54" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 34025 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 03:47:16.215564 22884 mvcc.cc:205] Tried to move back new op lower bound from 7315406996316450816 to 7315406996071862272. Current Snapshot: MvccSnapshot[applied={T|T < 7315406996316450816}]
I20260806 03:47:16.216956 22885 mvcc.cc:205] Tried to move back new op lower bound from 7315406996316450816 to 7315406996071862272. Current Snapshot: MvccSnapshot[applied={T|T < 7315406996316450816}]
I20260806 03:47:16.217784 22888 mvcc.cc:205] Tried to move back new op lower bound from 7315406996316450816 to 7315406996071862272. Current Snapshot: MvccSnapshot[applied={T|T < 7315406996316450816}]
I20260806 03:47:16.218938 21005 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42216:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 03:47:16.219045 21005 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42216:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 03:47:16.220520 21005 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 46d7cfc9ed574f1794039e138d8f6220: Sending DeleteTablet for 3 replicas of tablet 31bd1739b18f4080b8ca60699c3a43f6
I20260806 03:47:16.220857 22311 tablet_service.cc:1514] Processing DeleteTablet for tablet 31bd1739b18f4080b8ca60699c3a43f6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 03:47:16 UTC) from {username='oryx', principal='oryx/127.11.253.254@KRBTEST.COM'} at 127.0.0.1:44750
I20260806 03:47:16.220878 22702 tablet_service.cc:1514] Processing DeleteTablet for tablet 31bd1739b18f4080b8ca60699c3a43f6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 03:47:16 UTC) from {username='oryx', principal='oryx/127.11.253.254@KRBTEST.COM'} at 127.0.0.1:45240
I20260806 03:47:16.221060 22901 tablet_replica.cc:330] T 31bd1739b18f4080b8ca60699c3a43f6 P 21d27ac5b0aa4681b44bc0482b668d54: stopping tablet replica
I20260806 03:47:16.221158 22901 raft_consensus.cc:2229] T 31bd1739b18f4080b8ca60699c3a43f6 P 21d27ac5b0aa4681b44bc0482b668d54 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 03:47:16.221282 22901 raft_consensus.cc:2258] T 31bd1739b18f4080b8ca60699c3a43f6 P 21d27ac5b0aa4681b44bc0482b668d54 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 03:47:16.221570 22504 tablet_service.cc:1514] Processing DeleteTablet for tablet 31bd1739b18f4080b8ca60699c3a43f6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 03:47:16 UTC) from {username='oryx', principal='oryx/127.11.253.254@KRBTEST.COM'} at 127.0.0.1:38472
I20260806 03:47:16.221820 22903 tablet_replica.cc:330] T 31bd1739b18f4080b8ca60699c3a43f6 P 0fa8b770e150486aae31b4df6ad38a6e: stopping tablet replica
I20260806 03:47:16.221832 22901 ts_tablet_manager.cc:1905] T 31bd1739b18f4080b8ca60699c3a43f6 P 21d27ac5b0aa4681b44bc0482b668d54: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 03:47:16.222025 22903 raft_consensus.cc:2229] T 31bd1739b18f4080b8ca60699c3a43f6 P 0fa8b770e150486aae31b4df6ad38a6e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 03:47:16.222210 22903 raft_consensus.cc:2258] T 31bd1739b18f4080b8ca60699c3a43f6 P 0fa8b770e150486aae31b4df6ad38a6e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 03:47:16.222775 22903 ts_tablet_manager.cc:1905] T 31bd1739b18f4080b8ca60699c3a43f6 P 0fa8b770e150486aae31b4df6ad38a6e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 03:47:16.223284 12279 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskToVWRe/build/release/bin/kudu with pid 20957
I20260806 03:47:16.223770 22901 ts_tablet_manager.cc:1918] T 31bd1739b18f4080b8ca60699c3a43f6 P 21d27ac5b0aa4681b44bc0482b668d54: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 03:47:16.223860 22901 log.cc:1196] T 31bd1739b18f4080b8ca60699c3a43f6 P 21d27ac5b0aa4681b44bc0482b668d54: Deleting WAL directory at /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988004637995-12279-0/minicluster-data/ts-0/wal/wals/31bd1739b18f4080b8ca60699c3a43f6
I20260806 03:47:16.224184 22901 ts_tablet_manager.cc:1939] T 31bd1739b18f4080b8ca60699c3a43f6 P 21d27ac5b0aa4681b44bc0482b668d54: Deleting consensus metadata
I20260806 03:47:16.228240 22903 ts_tablet_manager.cc:1918] T 31bd1739b18f4080b8ca60699c3a43f6 P 0fa8b770e150486aae31b4df6ad38a6e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 03:47:16.228420 22903 log.cc:1196] T 31bd1739b18f4080b8ca60699c3a43f6 P 0fa8b770e150486aae31b4df6ad38a6e: Deleting WAL directory at /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988004637995-12279-0/minicluster-data/ts-1/wal/wals/31bd1739b18f4080b8ca60699c3a43f6
I20260806 03:47:16.228756 22903 ts_tablet_manager.cc:1939] T 31bd1739b18f4080b8ca60699c3a43f6 P 0fa8b770e150486aae31b4df6ad38a6e: Deleting consensus metadata
I20260806 03:47:16.230019 22902 tablet_replica.cc:330] T 31bd1739b18f4080b8ca60699c3a43f6 P 05f018b6c7cd4ee3a66404e6d7ef0729: stopping tablet replica
I20260806 03:47:16.230186 22902 raft_consensus.cc:2229] T 31bd1739b18f4080b8ca60699c3a43f6 P 05f018b6c7cd4ee3a66404e6d7ef0729 [term 1 LEADER]: Raft consensus shutting down.
I20260806 03:47:16.230331 22902 raft_consensus.cc:2258] T 31bd1739b18f4080b8ca60699c3a43f6 P 05f018b6c7cd4ee3a66404e6d7ef0729 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 03:47:16.231521 22902 ts_tablet_manager.cc:1905] T 31bd1739b18f4080b8ca60699c3a43f6 P 05f018b6c7cd4ee3a66404e6d7ef0729: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 03:47:16.233820 22902 ts_tablet_manager.cc:1918] T 31bd1739b18f4080b8ca60699c3a43f6 P 05f018b6c7cd4ee3a66404e6d7ef0729: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 03:47:16.233944 22902 log.cc:1196] T 31bd1739b18f4080b8ca60699c3a43f6 P 05f018b6c7cd4ee3a66404e6d7ef0729: Deleting WAL directory at /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988004637995-12279-0/minicluster-data/ts-2/wal/wals/31bd1739b18f4080b8ca60699c3a43f6
I20260806 03:47:16.234284 22902 ts_tablet_manager.cc:1939] T 31bd1739b18f4080b8ca60699c3a43f6 P 05f018b6c7cd4ee3a66404e6d7ef0729: Deleting consensus metadata
W20260806 03:47:16.238253 22440 connection.cc:535] server connection from 127.0.0.1:38472 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260806 03:47:16.238404 22639 connection.cc:535] server connection from 127.0.0.1:45240 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260806 03:47:16.238662 22233 connection.cc:535] server connection from 127.0.0.1:44750 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260806 03:47:16.248577 22826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:16.240265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39952 (local address 127.11.253.253:46317)
0806 03:47:16.240327 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0806 03:47:16.240329 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:47:16.240562 (+ 233us) server_negotiation.cc:407] Connection header received
0806 03:47:16.240679 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:47:16.240680 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:47:16.240696 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:47:16.240756 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:47:16.241737 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:16.242516 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:16.243323 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:16.243576 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:16.243669 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:16.245196 (+ 1527us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:47:16.245198 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:47:16.245198 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:47:16.245207 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:47:16.247239 (+ 2032us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:47:16.247621 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:47:16.247622 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:47:16.247623 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:47:16.247684 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:47:16.247928 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:47:16.247929 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:47:16.247929 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:47:16.248052 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:47:16.248160 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0806 03:47:16.248287 (+ 127us) server_negotiation.cc:299] Negotiation successful
0806 03:47:16.248386 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260806 03:47:16.249605 22830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:16.240412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44922 (local address 127.11.253.252:40283)
0806 03:47:16.240484 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0806 03:47:16.240487 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 03:47:16.240664 (+ 177us) server_negotiation.cc:407] Connection header received
0806 03:47:16.240777 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:47:16.240779 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:47:16.240792 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:47:16.240845 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:47:16.241744 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:16.242753 (+ 1009us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:16.243981 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:16.244337 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:16.244421 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:16.246074 (+ 1653us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:47:16.246075 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:47:16.246076 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:47:16.246084 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:47:16.248154 (+ 2070us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:47:16.248650 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:47:16.248652 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:47:16.248652 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:47:16.248726 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:47:16.248997 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:47:16.248998 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:47:16.248998 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:47:16.249129 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:47:16.249189 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0806 03:47:16.249380 (+ 191us) server_negotiation.cc:299] Negotiation successful
0806 03:47:16.249471 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260806 03:47:17.218154 22911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:17.217809 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:32833 (local address 127.11.253.193:52199)
0806 03:47:17.218000 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0806 03:47:17.218094 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.253.254:32833: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":112,"threads_started":1}
W20260806 03:47:17.218576 22399 heartbeater.cc:644] Failed to heartbeat to 127.11.253.254:32833 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.11.253.254:32833: connect: Connection refused (error 111)
I20260806 03:47:17.218883 22912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:17.218610 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:32833 (local address 127.11.253.194:50695)
0806 03:47:17.218757 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0806 03:47:17.218837 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.253.254:32833: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":78,"threads_started":1}
W20260806 03:47:17.219174 22602 heartbeater.cc:644] Failed to heartbeat to 127.11.253.254:32833 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.11.253.254:32833: connect: Connection refused (error 111)
I20260806 03:47:17.219259 22914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:17.218648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:32833 (local address 127.11.253.195:50213)
0806 03:47:17.219128 (+ 480us) negotiation.cc:107] Waiting for socket to connect
0806 03:47:17.219202 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.253.254:32833: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":461,"thread_start_us":99,"threads_started":1}
W20260806 03:47:17.219408 22804 heartbeater.cc:644] Failed to heartbeat to 127.11.253.254:32833 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.11.253.254:32833: connect: Connection refused (error 111)
I20260806 03:47:17.721299 22915 raft_consensus.cc:482] T 00000000000000000000000000000000 P 492ed5d42bfe4567aa936d6a80e98096 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 46d7cfc9ed574f1794039e138d8f6220)
I20260806 03:47:17.721439 22915 raft_consensus.cc:504] T 00000000000000000000000000000000 P 492ed5d42bfe4567aa936d6a80e98096 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "46d7cfc9ed574f1794039e138d8f6220" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 32833 } } peers { permanent_uuid: "85510334aee348ffbc06964d94b3087b" member_type: VOTER last_known_addr { host: "127.11.253.253" port: 46317 } attrs { promote: false } } peers { permanent_uuid: "492ed5d42bfe4567aa936d6a80e98096" member_type: VOTER last_known_addr { host: "127.11.253.252" port: 40283 } attrs { promote: false } }
I20260806 03:47:17.721782 22915 leader_election.cc:288] T 00000000000000000000000000000000 P 492ed5d42bfe4567aa936d6a80e98096 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 46d7cfc9ed574f1794039e138d8f6220 (127.11.253.254:32833), 85510334aee348ffbc06964d94b3087b (127.11.253.253:46317)
I20260806 03:47:17.722090 22916 raft_consensus.cc:482] T 00000000000000000000000000000000 P 85510334aee348ffbc06964d94b3087b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 46d7cfc9ed574f1794039e138d8f6220)
I20260806 03:47:17.722194 22918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:17.721951 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:32833 (local address 127.0.0.1:42698)
0806 03:47:17.722074 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0806 03:47:17.722147 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.253.254:32833: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":47,"threads_started":1}
W20260806 03:47:17.722330 22129 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.11.253.254:32833: connect: Connection refused (error 111)
I20260806 03:47:17.722374 22916 raft_consensus.cc:504] T 00000000000000000000000000000000 P 85510334aee348ffbc06964d94b3087b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "46d7cfc9ed574f1794039e138d8f6220" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 32833 } } peers { permanent_uuid: "85510334aee348ffbc06964d94b3087b" member_type: VOTER last_known_addr { host: "127.11.253.253" port: 46317 } attrs { promote: false } } peers { permanent_uuid: "492ed5d42bfe4567aa936d6a80e98096" member_type: VOTER last_known_addr { host: "127.11.253.252" port: 40283 } attrs { promote: false } }
I20260806 03:47:17.722746 22916 leader_election.cc:288] T 00000000000000000000000000000000 P 85510334aee348ffbc06964d94b3087b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 46d7cfc9ed574f1794039e138d8f6220 (127.11.253.254:32833), 492ed5d42bfe4567aa936d6a80e98096 (127.11.253.252:40283)
I20260806 03:47:17.722774 22918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:17.722653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:32833 (local address 127.0.0.1:42710)
0806 03:47:17.722710 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0806 03:47:17.722746 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.253.254:32833: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260806 03:47:17.723127 22129 leader_election.cc:334] T 00000000000000000000000000000000 P 492ed5d42bfe4567aa936d6a80e98096 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 46d7cfc9ed574f1794039e138d8f6220 (127.11.253.254:32833): Network error: Client connection negotiation failed: client connection to 127.11.253.254:32833: connect: Connection refused (error 111)
I20260806 03:47:17.723230 22924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:17.722815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:32833 (local address 127.0.0.1:42720)
0806 03:47:17.723151 (+ 336us) negotiation.cc:107] Waiting for socket to connect
0806 03:47:17.723192 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.253.254:32833: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":321,"thread_start_us":82,"threads_started":1}
W20260806 03:47:17.723330 21847 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.11.253.254:32833: connect: Connection refused (error 111)
I20260806 03:47:17.723767 22924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:17.723661 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:32833 (local address 127.0.0.1:42726)
0806 03:47:17.723706 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0806 03:47:17.723740 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.253.254:32833: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":20}
W20260806 03:47:17.723874 21847 leader_election.cc:334] T 00000000000000000000000000000000 P 85510334aee348ffbc06964d94b3087b [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 46d7cfc9ed574f1794039e138d8f6220 (127.11.253.254:32833): Network error: Client connection negotiation failed: client connection to 127.11.253.254:32833: connect: Connection refused (error 111)
I20260806 03:47:17.725289 22919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:17.722059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39956 (local address 127.11.253.253:46317)
0806 03:47:17.722194 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0806 03:47:17.722196 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:47:17.722292 (+ 96us) server_negotiation.cc:407] Connection header received
0806 03:47:17.722322 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:47:17.722323 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:47:17.722338 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:47:17.722421 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 03:47:17.722620 (+ 199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:17.723311 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:17.724536 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:17.724940 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:17.725032 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:17.725042 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0806 03:47:17.725183 (+ 141us) server_negotiation.cc:299] Negotiation successful
0806 03:47:17.725224 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113,"thread_start_us":70,"threads_started":1}
I20260806 03:47:17.725297 22917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:17.721963 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.253:46317 (local address 127.0.0.1:39956)
0806 03:47:17.722165 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0806 03:47:17.722179 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 03:47:17.722246 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:47:17.722489 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:47:17.722490 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:47:17.722493 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 03:47:17.722555 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:17.722558 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:17.723398 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:17.723399 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:17.724436 (+ 1037us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:17.724439 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:17.725031 (+ 592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:17.725032 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:17.725096 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:17.725100 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 03:47:17.725178 (+ 78us) client_negotiation.cc:240] Negotiation successful
0806 03:47:17.725229 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":53,"threads_started":1}
I20260806 03:47:17.725620 21890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "492ed5d42bfe4567aa936d6a80e98096" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "85510334aee348ffbc06964d94b3087b" is_pre_election: true
I20260806 03:47:17.725739 21890 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 85510334aee348ffbc06964d94b3087b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 492ed5d42bfe4567aa936d6a80e98096 in term 1.
I20260806 03:47:17.725916 22132 leader_election.cc:302] T 00000000000000000000000000000000 P 492ed5d42bfe4567aa936d6a80e98096 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 492ed5d42bfe4567aa936d6a80e98096, 85510334aee348ffbc06964d94b3087b; no voters: 46d7cfc9ed574f1794039e138d8f6220
I20260806 03:47:17.726054 22915 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 492ed5d42bfe4567aa936d6a80e98096 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260806 03:47:17.726104 22915 raft_consensus.cc:482] T 00000000000000000000000000000000 P 492ed5d42bfe4567aa936d6a80e98096 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 46d7cfc9ed574f1794039e138d8f6220)
I20260806 03:47:17.726131 22915 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 492ed5d42bfe4567aa936d6a80e98096 [term 1 FOLLOWER]: Advancing to term 2
I20260806 03:47:17.726364 22927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:17.723154 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44926 (local address 127.11.253.252:40283)
0806 03:47:17.723303 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0806 03:47:17.723305 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:47:17.723422 (+ 117us) server_negotiation.cc:407] Connection header received
0806 03:47:17.723523 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:47:17.723524 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:47:17.723534 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:47:17.723599 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 03:47:17.723749 (+ 150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:17.724397 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:17.725412 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:17.726002 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:17.726118 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:17.726127 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0806 03:47:17.726281 (+ 154us) server_negotiation.cc:299] Negotiation successful
0806 03:47:17.726315 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":93,"threads_started":1}
I20260806 03:47:17.726385 22923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:17.723023 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.252:40283 (local address 127.0.0.1:44926)
0806 03:47:17.723354 (+ 331us) negotiation.cc:107] Waiting for socket to connect
0806 03:47:17.723365 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 03:47:17.723467 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:47:17.723622 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:47:17.723623 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:47:17.723625 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 03:47:17.723675 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:17.723676 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:17.724497 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:17.724498 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:17.725267 (+ 769us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:17.725269 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:17.726124 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:17.726125 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:17.726195 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:17.726199 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 03:47:17.726268 (+ 69us) client_negotiation.cc:240] Negotiation successful
0806 03:47:17.726318 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":318,"thread_start_us":45,"threads_started":1}
I20260806 03:47:17.726663 22158 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "85510334aee348ffbc06964d94b3087b" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "492ed5d42bfe4567aa936d6a80e98096" is_pre_election: true
I20260806 03:47:17.727032 22915 raft_consensus.cc:504] T 00000000000000000000000000000000 P 492ed5d42bfe4567aa936d6a80e98096 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "46d7cfc9ed574f1794039e138d8f6220" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 32833 } } peers { permanent_uuid: "85510334aee348ffbc06964d94b3087b" member_type: VOTER last_known_addr { host: "127.11.253.253" port: 46317 } attrs { promote: false } } peers { permanent_uuid: "492ed5d42bfe4567aa936d6a80e98096" member_type: VOTER last_known_addr { host: "127.11.253.252" port: 40283 } attrs { promote: false } }
I20260806 03:47:17.727151 22158 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 492ed5d42bfe4567aa936d6a80e98096 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85510334aee348ffbc06964d94b3087b in current term 2: Already voted for candidate 492ed5d42bfe4567aa936d6a80e98096 in this term.
I20260806 03:47:17.727186 22915 leader_election.cc:288] T 00000000000000000000000000000000 P 492ed5d42bfe4567aa936d6a80e98096 [CANDIDATE]: Term 2 election: Requested vote from peers 46d7cfc9ed574f1794039e138d8f6220 (127.11.253.254:32833), 85510334aee348ffbc06964d94b3087b (127.11.253.253:46317)
I20260806 03:47:17.727353 21847 leader_election.cc:302] T 00000000000000000000000000000000 P 85510334aee348ffbc06964d94b3087b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85510334aee348ffbc06964d94b3087b; no voters: 46d7cfc9ed574f1794039e138d8f6220, 492ed5d42bfe4567aa936d6a80e98096
I20260806 03:47:17.727360 21890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "492ed5d42bfe4567aa936d6a80e98096" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "85510334aee348ffbc06964d94b3087b"
I20260806 03:47:17.727452 21890 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 85510334aee348ffbc06964d94b3087b [term 1 FOLLOWER]: Advancing to term 2
I20260806 03:47:17.727638 22917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:17.727267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:32833 (local address 127.0.0.1:42742)
0806 03:47:17.727578 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0806 03:47:17.727611 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.253.254:32833: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":293}
I20260806 03:47:17.728081 22917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:17.727969 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:32833 (local address 127.0.0.1:42754)
0806 03:47:17.728027 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0806 03:47:17.728059 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.253.254:32833: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260806 03:47:17.728191 22129 leader_election.cc:334] T 00000000000000000000000000000000 P 492ed5d42bfe4567aa936d6a80e98096 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 46d7cfc9ed574f1794039e138d8f6220 (127.11.253.254:32833): Network error: Client connection negotiation failed: client connection to 127.11.253.254:32833: connect: Connection refused (error 111)
I20260806 03:47:17.728294 21890 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 85510334aee348ffbc06964d94b3087b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 492ed5d42bfe4567aa936d6a80e98096 in term 2.
I20260806 03:47:17.728386 22916 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 85510334aee348ffbc06964d94b3087b [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 03:47:17.728470 22132 leader_election.cc:302] T 00000000000000000000000000000000 P 492ed5d42bfe4567aa936d6a80e98096 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 492ed5d42bfe4567aa936d6a80e98096, 85510334aee348ffbc06964d94b3087b; no voters: 46d7cfc9ed574f1794039e138d8f6220
I20260806 03:47:17.728549 22915 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 492ed5d42bfe4567aa936d6a80e98096 [term 2 FOLLOWER]: Leader election won for term 2
I20260806 03:47:17.728681 22915 raft_consensus.cc:686] T 00000000000000000000000000000000 P 492ed5d42bfe4567aa936d6a80e98096 [term 2 LEADER]: Becoming Leader. State: Replica: 492ed5d42bfe4567aa936d6a80e98096, State: Running, Role: LEADER
I20260806 03:47:17.728771 22915 consensus_queue.cc:237] T 00000000000000000000000000000000 P 492ed5d42bfe4567aa936d6a80e98096 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "46d7cfc9ed574f1794039e138d8f6220" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 32833 } } peers { permanent_uuid: "85510334aee348ffbc06964d94b3087b" member_type: VOTER last_known_addr { host: "127.11.253.253" port: 46317 } attrs { promote: false } } peers { permanent_uuid: "492ed5d42bfe4567aa936d6a80e98096" member_type: VOTER last_known_addr { host: "127.11.253.252" port: 40283 } attrs { promote: false } }
I20260806 03:47:17.729157 22932 sys_catalog.cc:455] T 00000000000000000000000000000000 P 492ed5d42bfe4567aa936d6a80e98096 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 492ed5d42bfe4567aa936d6a80e98096. Latest consensus state: current_term: 2 leader_uuid: "492ed5d42bfe4567aa936d6a80e98096" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "46d7cfc9ed574f1794039e138d8f6220" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 32833 } } peers { permanent_uuid: "85510334aee348ffbc06964d94b3087b" member_type: VOTER last_known_addr { host: "127.11.253.253" port: 46317 } attrs { promote: false } } peers { permanent_uuid: "492ed5d42bfe4567aa936d6a80e98096" member_type: VOTER last_known_addr { host: "127.11.253.252" port: 40283 } attrs { promote: false } } }
I20260806 03:47:17.729264 22932 sys_catalog.cc:458] T 00000000000000000000000000000000 P 492ed5d42bfe4567aa936d6a80e98096 [sys.catalog]: This master's current role is: LEADER
I20260806 03:47:17.729478 22935 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 03:47:17.729931 22935 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 03:47:17.730019 22935 catalog_manager.cc:1259] Loaded cluster ID: 21e5bafe25394fe580e3c98318fe43c7
I20260806 03:47:17.730052 22935 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 03:47:17.730242 22935 catalog_manager.cc:1504] Loading token signing keys...
I20260806 03:47:17.730345 22935 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 492ed5d42bfe4567aa936d6a80e98096: Loaded TSK: 0
I20260806 03:47:17.730489 22935 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 03:47:17.743928 22147 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44922:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 03:47:17.745621 22147 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 03:47:17.746687 22917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:17.746588 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:32833 (local address 127.0.0.1:42764)
0806 03:47:17.746626 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0806 03:47:17.746663 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.253.254:32833: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":20}
I20260806 03:47:17.746755 21890 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 85510334aee348ffbc06964d94b3087b [term 2 FOLLOWER]: Refusing update from remote peer 492ed5d42bfe4567aa936d6a80e98096: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 2 index: 14. (index mismatch)
I20260806 03:47:17.746994 22917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:17.746907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:32833 (local address 127.0.0.1:42776)
0806 03:47:17.746949 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0806 03:47:17.746971 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.253.254:32833: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
I20260806 03:47:17.747067 22932 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 492ed5d42bfe4567aa936d6a80e98096 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85510334aee348ffbc06964d94b3087b" member_type: VOTER last_known_addr { host: "127.11.253.253" port: 46317 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260806 03:47:17.747213 22129 consensus_peers.cc:487] T 00000000000000000000000000000000 P 492ed5d42bfe4567aa936d6a80e98096 -> Peer 46d7cfc9ed574f1794039e138d8f6220 (127.11.253.254:32833): Couldn't send request to peer 46d7cfc9ed574f1794039e138d8f6220. Status: Network error: Client connection negotiation failed: client connection to 127.11.253.254:32833: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260806 03:47:17.747771 22916 sys_catalog.cc:455] T 00000000000000000000000000000000 P 85510334aee348ffbc06964d94b3087b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 492ed5d42bfe4567aa936d6a80e98096. Latest consensus state: current_term: 2 leader_uuid: "492ed5d42bfe4567aa936d6a80e98096" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "46d7cfc9ed574f1794039e138d8f6220" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 32833 } } peers { permanent_uuid: "85510334aee348ffbc06964d94b3087b" member_type: VOTER last_known_addr { host: "127.11.253.253" port: 46317 } attrs { promote: false } } peers { permanent_uuid: "492ed5d42bfe4567aa936d6a80e98096" member_type: VOTER last_known_addr { host: "127.11.253.252" port: 40283 } attrs { promote: false } } }
I20260806 03:47:17.747884 22916 sys_catalog.cc:458] T 00000000000000000000000000000000 P 85510334aee348ffbc06964d94b3087b [sys.catalog]: This master's current role is: FOLLOWER
I20260806 03:47:17.748294 22915 sys_catalog.cc:455] T 00000000000000000000000000000000 P 492ed5d42bfe4567aa936d6a80e98096 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "492ed5d42bfe4567aa936d6a80e98096" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "46d7cfc9ed574f1794039e138d8f6220" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 32833 } } peers { permanent_uuid: "85510334aee348ffbc06964d94b3087b" member_type: VOTER last_known_addr { host: "127.11.253.253" port: 46317 } attrs { promote: false } } peers { permanent_uuid: "492ed5d42bfe4567aa936d6a80e98096" member_type: VOTER last_known_addr { host: "127.11.253.252" port: 40283 } attrs { promote: false } } }
I20260806 03:47:17.748382 22915 sys_catalog.cc:458] T 00000000000000000000000000000000 P 492ed5d42bfe4567aa936d6a80e98096 [sys.catalog]: This master's current role is: LEADER
I20260806 03:47:17.748728 22916 sys_catalog.cc:455] T 00000000000000000000000000000000 P 85510334aee348ffbc06964d94b3087b [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "492ed5d42bfe4567aa936d6a80e98096" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "46d7cfc9ed574f1794039e138d8f6220" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 32833 } } peers { permanent_uuid: "85510334aee348ffbc06964d94b3087b" member_type: VOTER last_known_addr { host: "127.11.253.253" port: 46317 } attrs { promote: false } } peers { permanent_uuid: "492ed5d42bfe4567aa936d6a80e98096" member_type: VOTER last_known_addr { host: "127.11.253.252" port: 40283 } attrs { promote: false } } }
I20260806 03:47:17.748813 22916 sys_catalog.cc:458] T 00000000000000000000000000000000 P 85510334aee348ffbc06964d94b3087b [sys.catalog]: This master's current role is: FOLLOWER
I20260806 03:47:17.748835 22938 mvcc.cc:205] Tried to move back new op lower bound from 7315407002608435200 to 7315407002537697280. Current Snapshot: MvccSnapshot[applied={T|T < 7315407002608435200}]
I20260806 03:47:17.753671 22917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:17.750352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.195:33553 (local address 127.0.0.1:45248)
0806 03:47:17.750388 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0806 03:47:17.750400 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 03:47:17.750443 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:47:17.750808 (+ 365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:47:17.750809 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:47:17.750811 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 03:47:17.750877 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:17.750879 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:17.751827 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:17.751828 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:17.752607 (+ 779us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:17.752609 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:17.753451 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:17.753452 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:17.753511 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:17.753516 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 03:47:17.753546 (+ 30us) client_negotiation.cc:240] Negotiation successful
0806 03:47:17.753597 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":22}
I20260806 03:47:17.754057 22942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:17.750463 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45248 (local address 127.11.253.195:33553)
0806 03:47:17.750597 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0806 03:47:17.750599 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:47:17.750608 (+ 9us) server_negotiation.cc:407] Connection header received
0806 03:47:17.750639 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:47:17.750639 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:47:17.750652 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:47:17.750966 (+ 314us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 03:47:17.751013 (+ 47us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:17.751751 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:17.752701 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:17.753361 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:17.753906 (+ 545us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:17.753917 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0806 03:47:17.753937 (+ 20us) server_negotiation.cc:299] Negotiation successful
0806 03:47:17.753981 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108,"thread_start_us":74,"threads_started":1}
I20260806 03:47:17.754272 22702 tablet_service.cc:1467] Processing CreateTablet for tablet d839cd8a3da540d6ae632aa9bf287040 (DEFAULT_TABLE table=test-table [id=0b932fb1ec1641b5b996503fdffa6953]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 03:47:17.754426 22918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:17.750442 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.194:45307 (local address 127.0.0.1:38478)
0806 03:47:17.750756 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0806 03:47:17.750766 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 03:47:17.750808 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:47:17.751336 (+ 528us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:47:17.751337 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:47:17.751341 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 03:47:17.751423 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:17.751425 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:17.752398 (+ 973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:17.752399 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:17.753424 (+ 1025us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:17.753427 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:17.754234 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:17.754235 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:17.754301 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:17.754304 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 03:47:17.754334 (+ 30us) client_negotiation.cc:240] Negotiation successful
0806 03:47:17.754371 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":299,"mutex_wait_us":49}
I20260806 03:47:17.754437 22702 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d839cd8a3da540d6ae632aa9bf287040. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:47:17.754684 22944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:17.750936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38478 (local address 127.11.253.194:45307)
0806 03:47:17.751072 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0806 03:47:17.751074 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:47:17.751082 (+ 8us) server_negotiation.cc:407] Connection header received
0806 03:47:17.751107 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:47:17.751107 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:47:17.751140 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:47:17.751250 (+ 110us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 03:47:17.751502 (+ 252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:17.752296 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:17.753509 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:17.754131 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:17.754558 (+ 427us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:17.754569 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0806 03:47:17.754588 (+ 19us) server_negotiation.cc:299] Negotiation successful
0806 03:47:17.754627 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":68,"threads_started":1}
I20260806 03:47:17.754884 22945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:17.751062 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44758 (local address 127.11.253.193:34025)
0806 03:47:17.751195 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0806 03:47:17.751197 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:47:17.751402 (+ 205us) server_negotiation.cc:407] Connection header received
0806 03:47:17.751433 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:47:17.751434 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:47:17.751444 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:47:17.751495 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 03:47:17.751669 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:17.752389 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:17.753637 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:17.754235 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:17.754297 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:17.754305 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0806 03:47:17.754742 (+ 437us) server_negotiation.cc:299] Negotiation successful
0806 03:47:17.754802 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":63,"threads_started":1}
I20260806 03:47:17.755035 22943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:17.750552 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.193:34025 (local address 127.0.0.1:44758)
0806 03:47:17.751287 (+ 735us) negotiation.cc:107] Waiting for socket to connect
0806 03:47:17.751296 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 03:47:17.751340 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:47:17.751535 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:47:17.751536 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:47:17.751537 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 03:47:17.751591 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:17.751593 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:17.752475 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:17.752476 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:17.753555 (+ 1079us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:17.753557 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:17.754313 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:17.754314 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:17.754440 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:17.754445 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 03:47:17.754705 (+ 260us) client_negotiation.cc:240] Negotiation successful
0806 03:47:17.754749 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":715,"thread_start_us":98,"threads_started":1}
I20260806 03:47:17.755290 22311 tablet_service.cc:1467] Processing CreateTablet for tablet d839cd8a3da540d6ae632aa9bf287040 (DEFAULT_TABLE table=test-table [id=0b932fb1ec1641b5b996503fdffa6953]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 03:47:17.755470 22311 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d839cd8a3da540d6ae632aa9bf287040. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:47:17.756062 22947 tablet_bootstrap.cc:492] T d839cd8a3da540d6ae632aa9bf287040 P 05f018b6c7cd4ee3a66404e6d7ef0729: Bootstrap starting.
I20260806 03:47:17.756237 22504 tablet_service.cc:1467] Processing CreateTablet for tablet d839cd8a3da540d6ae632aa9bf287040 (DEFAULT_TABLE table=test-table [id=0b932fb1ec1641b5b996503fdffa6953]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 03:47:17.756359 22504 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d839cd8a3da540d6ae632aa9bf287040. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:47:17.756475 22947 tablet_bootstrap.cc:654] T d839cd8a3da540d6ae632aa9bf287040 P 05f018b6c7cd4ee3a66404e6d7ef0729: Neither blocks nor log segments found. Creating new log.
I20260806 03:47:17.756729 22948 tablet_bootstrap.cc:492] T d839cd8a3da540d6ae632aa9bf287040 P 21d27ac5b0aa4681b44bc0482b668d54: Bootstrap starting.
I20260806 03:47:17.757077 22948 tablet_bootstrap.cc:654] T d839cd8a3da540d6ae632aa9bf287040 P 21d27ac5b0aa4681b44bc0482b668d54: Neither blocks nor log segments found. Creating new log.
I20260806 03:47:17.757692 22952 tablet_bootstrap.cc:492] T d839cd8a3da540d6ae632aa9bf287040 P 0fa8b770e150486aae31b4df6ad38a6e: Bootstrap starting.
I20260806 03:47:17.757707 22948 tablet_bootstrap.cc:492] T d839cd8a3da540d6ae632aa9bf287040 P 21d27ac5b0aa4681b44bc0482b668d54: No bootstrap required, opened a new log
I20260806 03:47:17.757792 22948 ts_tablet_manager.cc:1397] T d839cd8a3da540d6ae632aa9bf287040 P 21d27ac5b0aa4681b44bc0482b668d54: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260806 03:47:17.758065 22948 raft_consensus.cc:348] T d839cd8a3da540d6ae632aa9bf287040 P 21d27ac5b0aa4681b44bc0482b668d54 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05f018b6c7cd4ee3a66404e6d7ef0729" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 33553 } } peers { permanent_uuid: "0fa8b770e150486aae31b4df6ad38a6e" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 45307 } } peers { permanent_uuid: "21d27ac5b0aa4681b44bc0482b668d54" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 34025 } }
I20260806 03:47:17.758172 22952 tablet_bootstrap.cc:654] T d839cd8a3da540d6ae632aa9bf287040 P 0fa8b770e150486aae31b4df6ad38a6e: Neither blocks nor log segments found. Creating new log.
I20260806 03:47:17.758188 22947 tablet_bootstrap.cc:492] T d839cd8a3da540d6ae632aa9bf287040 P 05f018b6c7cd4ee3a66404e6d7ef0729: No bootstrap required, opened a new log
I20260806 03:47:17.758244 22947 ts_tablet_manager.cc:1397] T d839cd8a3da540d6ae632aa9bf287040 P 05f018b6c7cd4ee3a66404e6d7ef0729: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 03:47:17.758378 22947 raft_consensus.cc:348] T d839cd8a3da540d6ae632aa9bf287040 P 05f018b6c7cd4ee3a66404e6d7ef0729 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05f018b6c7cd4ee3a66404e6d7ef0729" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 33553 } } peers { permanent_uuid: "0fa8b770e150486aae31b4df6ad38a6e" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 45307 } } peers { permanent_uuid: "21d27ac5b0aa4681b44bc0482b668d54" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 34025 } }
I20260806 03:47:17.758451 22947 raft_consensus.cc:374] T d839cd8a3da540d6ae632aa9bf287040 P 05f018b6c7cd4ee3a66404e6d7ef0729 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:47:17.758476 22947 raft_consensus.cc:729] T d839cd8a3da540d6ae632aa9bf287040 P 05f018b6c7cd4ee3a66404e6d7ef0729 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 05f018b6c7cd4ee3a66404e6d7ef0729, State: Initialized, Role: FOLLOWER
I20260806 03:47:17.758535 22947 consensus_queue.cc:260] T d839cd8a3da540d6ae632aa9bf287040 P 05f018b6c7cd4ee3a66404e6d7ef0729 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05f018b6c7cd4ee3a66404e6d7ef0729" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 33553 } } peers { permanent_uuid: "0fa8b770e150486aae31b4df6ad38a6e" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 45307 } } peers { permanent_uuid: "21d27ac5b0aa4681b44bc0482b668d54" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 34025 } }
I20260806 03:47:17.758495 22948 raft_consensus.cc:374] T d839cd8a3da540d6ae632aa9bf287040 P 21d27ac5b0aa4681b44bc0482b668d54 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:47:17.758594 22948 raft_consensus.cc:729] T d839cd8a3da540d6ae632aa9bf287040 P 21d27ac5b0aa4681b44bc0482b668d54 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 21d27ac5b0aa4681b44bc0482b668d54, State: Initialized, Role: FOLLOWER
I20260806 03:47:17.758641 22948 consensus_queue.cc:260] T d839cd8a3da540d6ae632aa9bf287040 P 21d27ac5b0aa4681b44bc0482b668d54 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05f018b6c7cd4ee3a66404e6d7ef0729" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 33553 } } peers { permanent_uuid: "0fa8b770e150486aae31b4df6ad38a6e" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 45307 } } peers { permanent_uuid: "21d27ac5b0aa4681b44bc0482b668d54" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 34025 } }
I20260806 03:47:17.758697 22947 ts_tablet_manager.cc:1428] T d839cd8a3da540d6ae632aa9bf287040 P 05f018b6c7cd4ee3a66404e6d7ef0729: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260806 03:47:17.758829 22948 ts_tablet_manager.cc:1428] T d839cd8a3da540d6ae632aa9bf287040 P 21d27ac5b0aa4681b44bc0482b668d54: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260806 03:47:17.759389 22952 tablet_bootstrap.cc:492] T d839cd8a3da540d6ae632aa9bf287040 P 0fa8b770e150486aae31b4df6ad38a6e: No bootstrap required, opened a new log
I20260806 03:47:17.759455 22952 ts_tablet_manager.cc:1397] T d839cd8a3da540d6ae632aa9bf287040 P 0fa8b770e150486aae31b4df6ad38a6e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 03:47:17.759608 22959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:17.758979 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:32833 (local address 127.11.253.195:37035)
0806 03:47:17.759523 (+ 544us) negotiation.cc:107] Waiting for socket to connect
0806 03:47:17.759572 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.253.254:32833: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":498,"thread_start_us":84,"threads_started":1}
I20260806 03:47:17.759601 22952 raft_consensus.cc:348] T d839cd8a3da540d6ae632aa9bf287040 P 0fa8b770e150486aae31b4df6ad38a6e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05f018b6c7cd4ee3a66404e6d7ef0729" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 33553 } } peers { permanent_uuid: "0fa8b770e150486aae31b4df6ad38a6e" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 45307 } } peers { permanent_uuid: "21d27ac5b0aa4681b44bc0482b668d54" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 34025 } }
I20260806 03:47:17.759677 22952 raft_consensus.cc:374] T d839cd8a3da540d6ae632aa9bf287040 P 0fa8b770e150486aae31b4df6ad38a6e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:47:17.759703 22952 raft_consensus.cc:729] T d839cd8a3da540d6ae632aa9bf287040 P 0fa8b770e150486aae31b4df6ad38a6e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0fa8b770e150486aae31b4df6ad38a6e, State: Initialized, Role: FOLLOWER
I20260806 03:47:17.759768 22952 consensus_queue.cc:260] T d839cd8a3da540d6ae632aa9bf287040 P 0fa8b770e150486aae31b4df6ad38a6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05f018b6c7cd4ee3a66404e6d7ef0729" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 33553 } } peers { permanent_uuid: "0fa8b770e150486aae31b4df6ad38a6e" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 45307 } } peers { permanent_uuid: "21d27ac5b0aa4681b44bc0482b668d54" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 34025 } }
I20260806 03:47:17.759883 22397 heartbeater.cc:503] Master 127.11.253.252:40283 was elected leader, sending a full tablet report...
I20260806 03:47:17.759940 22952 ts_tablet_manager.cc:1428] T d839cd8a3da540d6ae632aa9bf287040 P 0fa8b770e150486aae31b4df6ad38a6e: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260806 03:47:17.760166 22960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:17.759094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:32833 (local address 127.11.253.193:43633)
0806 03:47:17.760071 (+ 977us) negotiation.cc:107] Waiting for socket to connect
0806 03:47:17.760114 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.253.254:32833: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":954,"thread_start_us":77,"threads_started":1}
I20260806 03:47:17.760336 22803 heartbeater.cc:503] Master 127.11.253.252:40283 was elected leader, sending a full tablet report...
I20260806 03:47:17.760569 22600 heartbeater.cc:503] Master 127.11.253.252:40283 was elected leader, sending a full tablet report...
I20260806 03:47:17.761025 22960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:17.760700 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:32833 (local address 127.11.253.193:39129)
0806 03:47:17.760926 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0806 03:47:17.760958 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.253.254:32833: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":206}
I20260806 03:47:17.761129 22959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:17.761002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:32833 (local address 127.11.253.195:58547)
0806 03:47:17.761049 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0806 03:47:17.761078 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.253.254:32833: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
I20260806 03:47:17.761348 22963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:17.760149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:32833 (local address 127.11.253.194:53353)
0806 03:47:17.761270 (+ 1121us) negotiation.cc:107] Waiting for socket to connect
0806 03:47:17.761310 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.253.254:32833: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1098,"thread_start_us":80,"threads_started":1}
I20260806 03:47:17.761672 22963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:17.761574 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:32833 (local address 127.11.253.194:34731)
0806 03:47:17.761626 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0806 03:47:17.761650 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.253.254:32833: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260806 03:47:17.826325 22962 raft_consensus.cc:482] T d839cd8a3da540d6ae632aa9bf287040 P 0fa8b770e150486aae31b4df6ad38a6e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 03:47:17.826448 22962 raft_consensus.cc:504] T d839cd8a3da540d6ae632aa9bf287040 P 0fa8b770e150486aae31b4df6ad38a6e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05f018b6c7cd4ee3a66404e6d7ef0729" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 33553 } } peers { permanent_uuid: "0fa8b770e150486aae31b4df6ad38a6e" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 45307 } } peers { permanent_uuid: "21d27ac5b0aa4681b44bc0482b668d54" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 34025 } }
I20260806 03:47:17.826908 22962 leader_election.cc:288] T d839cd8a3da540d6ae632aa9bf287040 P 0fa8b770e150486aae31b4df6ad38a6e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 05f018b6c7cd4ee3a66404e6d7ef0729 (127.11.253.195:33553), 21d27ac5b0aa4681b44bc0482b668d54 (127.11.253.193:34025)
I20260806 03:47:17.831305 22942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:17.827352 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.253.194:53289 (local address 127.11.253.195:33553)
0806 03:47:17.827400 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0806 03:47:17.827402 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:47:17.827573 (+ 171us) server_negotiation.cc:407] Connection header received
0806 03:47:17.827615 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:47:17.827616 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:47:17.827624 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:47:17.827666 (+ 42us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 03:47:17.827827 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:17.828523 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:17.830322 (+ 1799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:17.830910 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:17.830989 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:17.831000 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0806 03:47:17.831182 (+ 182us) server_negotiation.cc:299] Negotiation successful
0806 03:47:17.831228 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260806 03:47:17.831350 22945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:17.827252 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.253.194:48993 (local address 127.11.253.193:34025)
0806 03:47:17.827387 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0806 03:47:17.827389 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:47:17.827401 (+ 12us) server_negotiation.cc:407] Connection header received
0806 03:47:17.827434 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:47:17.827435 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:47:17.827447 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:47:17.827511 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 03:47:17.827729 (+ 218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:17.828522 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:17.830321 (+ 1799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:17.830906 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:17.830989 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:17.831000 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0806 03:47:17.831190 (+ 190us) server_negotiation.cc:299] Negotiation successful
0806 03:47:17.831229 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97}
I20260806 03:47:17.831418 22963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:17.827140 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.193:34025 (local address 127.11.253.194:48993)
0806 03:47:17.827217 (+ 77us) negotiation.cc:107] Waiting for socket to connect
0806 03:47:17.827232 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 03:47:17.827307 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:47:17.827549 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:47:17.827550 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:47:17.827552 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 03:47:17.827643 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:17.827645 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:17.828696 (+ 1051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:17.828697 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:17.830154 (+ 1457us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:17.830158 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:17.831007 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:17.831008 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:17.831088 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:17.831093 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 03:47:17.831164 (+ 71us) client_negotiation.cc:240] Negotiation successful
0806 03:47:17.831306 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":49,"mutex_wait_us":205}
I20260806 03:47:17.831434 22964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:17.827141 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.195:33553 (local address 127.11.253.194:53289)
0806 03:47:17.827471 (+ 330us) negotiation.cc:107] Waiting for socket to connect
0806 03:47:17.827478 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0806 03:47:17.827552 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:47:17.827683 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:47:17.827684 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:47:17.827686 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 03:47:17.827735 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:17.827736 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:17.828693 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:17.828695 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:17.830154 (+ 1459us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:17.830158 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:17.831046 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:17.831047 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:17.831104 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:17.831107 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 03:47:17.831167 (+ 60us) client_negotiation.cc:240] Negotiation successful
0806 03:47:17.831306 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":311,"mutex_wait_us":145,"thread_start_us":173,"threads_started":1}
I20260806 03:47:17.831781 22738 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d839cd8a3da540d6ae632aa9bf287040" candidate_uuid: "0fa8b770e150486aae31b4df6ad38a6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05f018b6c7cd4ee3a66404e6d7ef0729" is_pre_election: true
I20260806 03:47:17.831785 22339 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d839cd8a3da540d6ae632aa9bf287040" candidate_uuid: "0fa8b770e150486aae31b4df6ad38a6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "21d27ac5b0aa4681b44bc0482b668d54" is_pre_election: true
I20260806 03:47:17.831938 22339 raft_consensus.cc:2454] T d839cd8a3da540d6ae632aa9bf287040 P 21d27ac5b0aa4681b44bc0482b668d54 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0fa8b770e150486aae31b4df6ad38a6e in term 0.
I20260806 03:47:17.831946 22738 raft_consensus.cc:2454] T d839cd8a3da540d6ae632aa9bf287040 P 05f018b6c7cd4ee3a66404e6d7ef0729 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0fa8b770e150486aae31b4df6ad38a6e in term 0.
I20260806 03:47:17.832159 22441 leader_election.cc:302] T d839cd8a3da540d6ae632aa9bf287040 P 0fa8b770e150486aae31b4df6ad38a6e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0fa8b770e150486aae31b4df6ad38a6e, 21d27ac5b0aa4681b44bc0482b668d54; no voters:
I20260806 03:47:17.832329 22962 raft_consensus.cc:2781] T d839cd8a3da540d6ae632aa9bf287040 P 0fa8b770e150486aae31b4df6ad38a6e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 03:47:17.832383 22962 raft_consensus.cc:482] T d839cd8a3da540d6ae632aa9bf287040 P 0fa8b770e150486aae31b4df6ad38a6e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 03:47:17.832404 22962 raft_consensus.cc:3037] T d839cd8a3da540d6ae632aa9bf287040 P 0fa8b770e150486aae31b4df6ad38a6e [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:47:17.833191 22962 raft_consensus.cc:504] T d839cd8a3da540d6ae632aa9bf287040 P 0fa8b770e150486aae31b4df6ad38a6e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05f018b6c7cd4ee3a66404e6d7ef0729" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 33553 } } peers { permanent_uuid: "0fa8b770e150486aae31b4df6ad38a6e" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 45307 } } peers { permanent_uuid: "21d27ac5b0aa4681b44bc0482b668d54" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 34025 } }
I20260806 03:47:17.833350 22962 leader_election.cc:288] T d839cd8a3da540d6ae632aa9bf287040 P 0fa8b770e150486aae31b4df6ad38a6e [CANDIDATE]: Term 1 election: Requested vote from peers 05f018b6c7cd4ee3a66404e6d7ef0729 (127.11.253.195:33553), 21d27ac5b0aa4681b44bc0482b668d54 (127.11.253.193:34025)
I20260806 03:47:17.833493 22738 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d839cd8a3da540d6ae632aa9bf287040" candidate_uuid: "0fa8b770e150486aae31b4df6ad38a6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05f018b6c7cd4ee3a66404e6d7ef0729"
I20260806 03:47:17.833531 22339 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d839cd8a3da540d6ae632aa9bf287040" candidate_uuid: "0fa8b770e150486aae31b4df6ad38a6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "21d27ac5b0aa4681b44bc0482b668d54"
I20260806 03:47:17.833586 22738 raft_consensus.cc:3037] T d839cd8a3da540d6ae632aa9bf287040 P 05f018b6c7cd4ee3a66404e6d7ef0729 [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:47:17.833602 22339 raft_consensus.cc:3037] T d839cd8a3da540d6ae632aa9bf287040 P 21d27ac5b0aa4681b44bc0482b668d54 [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:47:17.834342 22738 raft_consensus.cc:2454] T d839cd8a3da540d6ae632aa9bf287040 P 05f018b6c7cd4ee3a66404e6d7ef0729 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0fa8b770e150486aae31b4df6ad38a6e in term 1.
I20260806 03:47:17.834326 22339 raft_consensus.cc:2454] T d839cd8a3da540d6ae632aa9bf287040 P 21d27ac5b0aa4681b44bc0482b668d54 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0fa8b770e150486aae31b4df6ad38a6e in term 1.
I20260806 03:47:17.834559 22444 leader_election.cc:302] T d839cd8a3da540d6ae632aa9bf287040 P 0fa8b770e150486aae31b4df6ad38a6e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05f018b6c7cd4ee3a66404e6d7ef0729, 0fa8b770e150486aae31b4df6ad38a6e; no voters:
I20260806 03:47:17.834645 22962 raft_consensus.cc:2781] T d839cd8a3da540d6ae632aa9bf287040 P 0fa8b770e150486aae31b4df6ad38a6e [term 1 FOLLOWER]: Leader election won for term 1
I20260806 03:47:17.834815 22962 raft_consensus.cc:686] T d839cd8a3da540d6ae632aa9bf287040 P 0fa8b770e150486aae31b4df6ad38a6e [term 1 LEADER]: Becoming Leader. State: Replica: 0fa8b770e150486aae31b4df6ad38a6e, State: Running, Role: LEADER
I20260806 03:47:17.834878 22962 consensus_queue.cc:237] T d839cd8a3da540d6ae632aa9bf287040 P 0fa8b770e150486aae31b4df6ad38a6e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05f018b6c7cd4ee3a66404e6d7ef0729" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 33553 } } peers { permanent_uuid: "0fa8b770e150486aae31b4df6ad38a6e" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 45307 } } peers { permanent_uuid: "21d27ac5b0aa4681b44bc0482b668d54" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 34025 } }
I20260806 03:47:17.835191 22964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:17.835071 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:32833 (local address 127.11.253.194:52371)
0806 03:47:17.835127 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0806 03:47:17.835158 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.253.254:32833: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260806 03:47:17.835889 22148 catalog_manager.cc:5495] T d839cd8a3da540d6ae632aa9bf287040 P 0fa8b770e150486aae31b4df6ad38a6e reported cstate change: term changed from 0 to 1, leader changed from <none> to 0fa8b770e150486aae31b4df6ad38a6e (127.11.253.194). New cstate: current_term: 1 leader_uuid: "0fa8b770e150486aae31b4df6ad38a6e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05f018b6c7cd4ee3a66404e6d7ef0729" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 33553 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0fa8b770e150486aae31b4df6ad38a6e" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 45307 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "21d27ac5b0aa4681b44bc0482b668d54" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 34025 } health_report { overall_health: UNKNOWN } } }
I20260806 03:47:17.846799 22944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:17.844307 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38488 (local address 127.11.253.194:45307)
0806 03:47:17.844371 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0806 03:47:17.844373 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:47:17.844381 (+ 8us) server_negotiation.cc:407] Connection header received
0806 03:47:17.844415 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:47:17.844416 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:47:17.844468 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:47:17.844584 (+ 116us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 03:47:17.844695 (+ 111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:17.845163 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:17.846013 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:17.846207 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:17.846292 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:17.846466 (+ 174us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 03:47:17.846571 (+ 105us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 03:47:17.846654 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0806 03:47:17.846696 (+ 42us) server_negotiation.cc:299] Negotiation successful
0806 03:47:17.846717 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260806 03:47:17.847884 22339 raft_consensus.cc:1261] T d839cd8a3da540d6ae632aa9bf287040 P 21d27ac5b0aa4681b44bc0482b668d54 [term 1 FOLLOWER]: Refusing update from remote peer 0fa8b770e150486aae31b4df6ad38a6e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 03:47:17.848009 22738 raft_consensus.cc:1261] T d839cd8a3da540d6ae632aa9bf287040 P 05f018b6c7cd4ee3a66404e6d7ef0729 [term 1 FOLLOWER]: Refusing update from remote peer 0fa8b770e150486aae31b4df6ad38a6e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 03:47:17.848170 22965 consensus_queue.cc:1035] T d839cd8a3da540d6ae632aa9bf287040 P 0fa8b770e150486aae31b4df6ad38a6e [LEADER]: Connected to new peer: Peer: permanent_uuid: "21d27ac5b0aa4681b44bc0482b668d54" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 34025 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 03:47:17.848282 22962 consensus_queue.cc:1035] T d839cd8a3da540d6ae632aa9bf287040 P 0fa8b770e150486aae31b4df6ad38a6e [LEADER]: Connected to new peer: Peer: permanent_uuid: "05f018b6c7cd4ee3a66404e6d7ef0729" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 33553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 03:47:17.849243 22960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:17.848928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:32833 (local address 127.11.253.193:54815)
0806 03:47:17.849118 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0806 03:47:17.849153 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.253.254:32833: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":169}
I20260806 03:47:17.849547 22959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:17.849394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:32833 (local address 127.11.253.195:35725)
0806 03:47:17.849482 (+ 88us) negotiation.cc:107] Waiting for socket to connect
0806 03:47:17.849511 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.253.254:32833: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260806 03:47:17.851100 22959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:17.850981 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:32833 (local address 127.11.253.195:43939)
0806 03:47:17.851024 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0806 03:47:17.851052 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.253.254:32833: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260806 03:47:17.851249 22960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:17.850981 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:32833 (local address 127.11.253.193:52767)
0806 03:47:17.851187 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0806 03:47:17.851219 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.253.254:32833: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":190}
I20260806 03:47:17.851742 22964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:17.850252 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:32833 (local address 127.11.253.194:39441)
0806 03:47:17.851666 (+ 1414us) negotiation.cc:107] Waiting for socket to connect
0806 03:47:17.851709 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.253.254:32833: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1400}
I20260806 03:47:17.852476 22964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:17.852378 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:32833 (local address 127.11.253.194:44445)
0806 03:47:17.852419 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0806 03:47:17.852445 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.253.254:32833: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
I20260806 03:47:17.852952 22148 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44922:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 03:47:17.853039 22148 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44922:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 03:47:17.854480 22148 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 492ed5d42bfe4567aa936d6a80e98096: Sending DeleteTablet for 3 replicas of tablet d839cd8a3da540d6ae632aa9bf287040
I20260806 03:47:17.854816 22702 tablet_service.cc:1514] Processing DeleteTablet for tablet d839cd8a3da540d6ae632aa9bf287040 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 03:47:17 UTC) from {username='oryx', principal='oryx/127.11.253.252@KRBTEST.COM'} at 127.0.0.1:45248
I20260806 03:47:17.854839 22311 tablet_service.cc:1514] Processing DeleteTablet for tablet d839cd8a3da540d6ae632aa9bf287040 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 03:47:17 UTC) from {username='oryx', principal='oryx/127.11.253.252@KRBTEST.COM'} at 127.0.0.1:44758
I20260806 03:47:17.855031 22504 tablet_service.cc:1514] Processing DeleteTablet for tablet d839cd8a3da540d6ae632aa9bf287040 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 03:47:17 UTC) from {username='oryx', principal='oryx/127.11.253.252@KRBTEST.COM'} at 127.0.0.1:38478
I20260806 03:47:17.855360 22986 tablet_replica.cc:330] T d839cd8a3da540d6ae632aa9bf287040 P 21d27ac5b0aa4681b44bc0482b668d54: stopping tablet replica
I20260806 03:47:17.855423 22986 raft_consensus.cc:2229] T d839cd8a3da540d6ae632aa9bf287040 P 21d27ac5b0aa4681b44bc0482b668d54 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 03:47:17.855417 12279 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskToVWRe/build/release/bin/kudu with pid 22210
I20260806 03:47:17.855470 22986 raft_consensus.cc:2258] T d839cd8a3da540d6ae632aa9bf287040 P 21d27ac5b0aa4681b44bc0482b668d54 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 03:47:17.859896 22987 tablet_replica.cc:330] T d839cd8a3da540d6ae632aa9bf287040 P 0fa8b770e150486aae31b4df6ad38a6e: stopping tablet replica
I20260806 03:47:17.859979 22987 raft_consensus.cc:2229] T d839cd8a3da540d6ae632aa9bf287040 P 0fa8b770e150486aae31b4df6ad38a6e [term 1 LEADER]: Raft consensus shutting down.
I20260806 03:47:17.860077 22987 raft_consensus.cc:2258] T d839cd8a3da540d6ae632aa9bf287040 P 0fa8b770e150486aae31b4df6ad38a6e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 03:47:17.861383 22984 tablet_replica.cc:330] T d839cd8a3da540d6ae632aa9bf287040 P 05f018b6c7cd4ee3a66404e6d7ef0729: stopping tablet replica
I20260806 03:47:17.861467 22984 raft_consensus.cc:2229] T d839cd8a3da540d6ae632aa9bf287040 P 05f018b6c7cd4ee3a66404e6d7ef0729 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 03:47:17.861544 22984 raft_consensus.cc:2258] T d839cd8a3da540d6ae632aa9bf287040 P 05f018b6c7cd4ee3a66404e6d7ef0729 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 03:47:17.861665 22987 ts_tablet_manager.cc:1905] T d839cd8a3da540d6ae632aa9bf287040 P 0fa8b770e150486aae31b4df6ad38a6e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 03:47:17.862555 22984 ts_tablet_manager.cc:1905] T d839cd8a3da540d6ae632aa9bf287040 P 05f018b6c7cd4ee3a66404e6d7ef0729: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 03:47:17.863560 22987 ts_tablet_manager.cc:1918] T d839cd8a3da540d6ae632aa9bf287040 P 0fa8b770e150486aae31b4df6ad38a6e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 03:47:17.863696 22987 log.cc:1196] T d839cd8a3da540d6ae632aa9bf287040 P 0fa8b770e150486aae31b4df6ad38a6e: Deleting WAL directory at /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988004637995-12279-0/minicluster-data/ts-1/wal/wals/d839cd8a3da540d6ae632aa9bf287040
I20260806 03:47:17.864037 22987 ts_tablet_manager.cc:1939] T d839cd8a3da540d6ae632aa9bf287040 P 0fa8b770e150486aae31b4df6ad38a6e: Deleting consensus metadata
I20260806 03:47:17.864508 22129 catalog_manager.cc:4841] TS 0fa8b770e150486aae31b4df6ad38a6e (127.11.253.194:45307): tablet d839cd8a3da540d6ae632aa9bf287040 (table test-table [id=0b932fb1ec1641b5b996503fdffa6953]) successfully deleted
I20260806 03:47:17.865003 22984 ts_tablet_manager.cc:1918] T d839cd8a3da540d6ae632aa9bf287040 P 05f018b6c7cd4ee3a66404e6d7ef0729: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 03:47:17.865091 22984 log.cc:1196] T d839cd8a3da540d6ae632aa9bf287040 P 05f018b6c7cd4ee3a66404e6d7ef0729: Deleting WAL directory at /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988004637995-12279-0/minicluster-data/ts-2/wal/wals/d839cd8a3da540d6ae632aa9bf287040
I20260806 03:47:17.865409 22984 ts_tablet_manager.cc:1939] T d839cd8a3da540d6ae632aa9bf287040 P 05f018b6c7cd4ee3a66404e6d7ef0729: Deleting consensus metadata
I20260806 03:47:17.865759 22133 catalog_manager.cc:4841] TS 05f018b6c7cd4ee3a66404e6d7ef0729 (127.11.253.195:33553): tablet d839cd8a3da540d6ae632aa9bf287040 (table test-table [id=0b932fb1ec1641b5b996503fdffa6953]) successfully deleted
W20260806 03:47:17.870113 22943 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 03:47:17.869942 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.193:34025 (local address 127.0.0.1:44760)
0806 03:47:17.869984 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0806 03:47:17.869997 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 03:47:17.870057 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.253.193:34025: BlockingWrite error: write error: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":20}
W20260806 03:47:17.870290 22131 catalog_manager.cc:4584] TS 21d27ac5b0aa4681b44bc0482b668d54 (127.11.253.193:34025): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet d839cd8a3da540d6ae632aa9bf287040: Network error: Client connection negotiation failed: client connection to 127.11.253.193:34025: BlockingWrite error: write error: Connection reset by peer (error 104)
I20260806 03:47:17.870375 12279 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskToVWRe/build/release/bin/kudu with pid 22411
I20260806 03:47:17.881387 12279 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskToVWRe/build/release/bin/kudu with pid 22608
I20260806 03:47:17.895220 12279 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskToVWRe/build/release/bin/kudu with pid 21667
I20260806 03:47:17.905047 12279 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskToVWRe/build/release/bin/kudu with pid 21948
2026-08-06T03:47:17Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (5189 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785988004637995-12279-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 03:47:17 dist-test-slave-6ntk krb5kdc[22990](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 03:47:17 dist-test-slave-6ntk krb5kdc[22990](info): set up 2 sockets
Aug 06 03:47:17 dist-test-slave-6ntk krb5kdc[22990](info): commencing operation
krb5kdc: starting...
W20260806 03:47:19.161602 12279 mini_kdc.cc:121] Time spent starting KDC: real 1.226s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785988004637995-12279-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785988004637995-12279-0/krb5kdc/test-user.keytab.
Aug 06 03:47:19 dist-test-slave-6ntk krb5kdc[22990](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988039, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T03:47:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T03:47:19Z Disabled control of system clock
WARNING: no policy specified for oryx/127.11.253.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.11.253.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.11.253.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785988004637995-12279-0/krb5kdc/oryx_127.11.253.254.keytab.
Entry for principal oryx/127.11.253.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785988004637995-12279-0/krb5kdc/oryx_127.11.253.254.keytab.
I20260806 03:47:19.293550 12279 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskToVWRe/build/release/bin/kudu
/tmp/dist-test-taskToVWRe/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785988004637995-12279-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785988004637995-12279-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785988004637995-12279-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785988004637995-12279-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.253.254:38857
--webserver_interface=127.11.253.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785988004637995-12279-0/krb5kdc/oryx_127.11.253.254.keytab
--principal=oryx/127.11.253.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.253.212:35693
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.253.254:38857
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785988004637995-12279-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785988004637995-12279-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785988004637995-12279-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:47:19.378623 23007 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:47:19.378878 23007 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:47:19.378919 23007 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:47:19.380060 23007 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 03:47:19.380127 23007 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260806 03:47:19.380148 23007 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:47:19.380174 23007 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:47:19.380206 23007 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 03:47:19.380228 23007 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 03:47:19.381733 23007 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.253.212:35693
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785988004637995-12279-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785988004637995-12279-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.253.254:38857
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785988004637995-12279-0/krb5kdc/oryx_127.11.253.254.keytab
--principal=oryx/127.11.253.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.253.254:38857
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785988004637995-12279-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.253.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23007
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785988004637995-12279-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:47:19.382037 23007 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:47:19.382267 23007 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:47:19.387238 23016 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:47:19.387410 23013 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:47:19.387444 23014 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:47:19.387444 23007 server_base.cc:1034] running on GCE node
I20260806 03:47:19.387776 23007 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:47:19.388101 23007 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:47:19.389266 23007 hybrid_clock.cc:648] HybridClock initialized: now 1785988039389232 us; error 52 us; skew 500 ppm
Aug 06 03:47:19 dist-test-slave-6ntk krb5kdc[22990](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988039, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.253.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:47:19.391769 23007 init.cc:377] Logged in from keytab as oryx/127.11.253.254@KRBTEST.COM (short username oryx)
I20260806 03:47:19.392365 23007 webserver.cc:460] Webserver started at http://127.11.253.254:46547/ using document root <none> and password file <none>
I20260806 03:47:19.392601 23007 fs_manager.cc:359] Metadata directory not provided
I20260806 03:47:19.392670 23007 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:47:19.392836 23007 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:47:19.393996 23007 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785988004637995-12279-0/minicluster-data/master-0/data/instance:
uuid: "1faf2e28f8564261b4b003157ec7c205"
format_stamp: "Formatted at 2026-08-06 03:47:19 on dist-test-slave-6ntk"
server_key: "e5f3d2a846ab054519a1815058b621f1"
server_key_iv: "6a15ba3c77fbcff56d53bfe202051439"
server_key_version: "encryptionkey@0"
I20260806 03:47:19.394372 23007 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785988004637995-12279-0/minicluster-data/master-0/wal/instance:
uuid: "1faf2e28f8564261b4b003157ec7c205"
format_stamp: "Formatted at 2026-08-06 03:47:19 on dist-test-slave-6ntk"
server_key: "e5f3d2a846ab054519a1815058b621f1"
server_key_iv: "6a15ba3c77fbcff56d53bfe202051439"
server_key_version: "encryptionkey@0"
I20260806 03:47:19.395807 23007 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260806 03:47:19.396538 23024 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:47:19.396719 23007 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 03:47:19.396849 23007 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785988004637995-12279-0/minicluster-data/master-0/data,/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785988004637995-12279-0/minicluster-data/master-0/wal
uuid: "1faf2e28f8564261b4b003157ec7c205"
format_stamp: "Formatted at 2026-08-06 03:47:19 on dist-test-slave-6ntk"
server_key: "e5f3d2a846ab054519a1815058b621f1"
server_key_iv: "6a15ba3c77fbcff56d53bfe202051439"
server_key_version: "encryptionkey@0"
I20260806 03:47:19.396926 23007 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785988004637995-12279-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785988004637995-12279-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785988004637995-12279-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:47:19.414482 23007 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:47:19.416440 23007 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:47:19.416610 23007 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:47:19.421844 23007 rpc_server.cc:307] RPC server started. Bound to: 127.11.253.254:38857
I20260806 03:47:19.421934 23118 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.253.254:38857 every 8 connection(s)
I20260806 03:47:19.422287 23007 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785988004637995-12279-0/minicluster-data/master-0/data/info.pb
I20260806 03:47:19.422986 23119 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:47:19.425299 23119 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1faf2e28f8564261b4b003157ec7c205: Bootstrap starting.
I20260806 03:47:19.425981 23119 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1faf2e28f8564261b4b003157ec7c205: Neither blocks nor log segments found. Creating new log.
I20260806 03:47:19.426290 23119 log.cc:824] T 00000000000000000000000000000000 P 1faf2e28f8564261b4b003157ec7c205: Log is configured to *not* fsync() on all Append() calls
I20260806 03:47:19.426992 23119 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1faf2e28f8564261b4b003157ec7c205: No bootstrap required, opened a new log
I20260806 03:47:19.429065 23119 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1faf2e28f8564261b4b003157ec7c205 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1faf2e28f8564261b4b003157ec7c205" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 38857 } }
I20260806 03:47:19.429206 23119 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1faf2e28f8564261b4b003157ec7c205 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:47:19.429229 23119 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1faf2e28f8564261b4b003157ec7c205 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1faf2e28f8564261b4b003157ec7c205, State: Initialized, Role: FOLLOWER
I20260806 03:47:19.429378 23119 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1faf2e28f8564261b4b003157ec7c205 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1faf2e28f8564261b4b003157ec7c205" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 38857 } }
I20260806 03:47:19.429453 23119 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1faf2e28f8564261b4b003157ec7c205 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 03:47:19.429492 23119 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1faf2e28f8564261b4b003157ec7c205 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 03:47:19.429523 23119 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1faf2e28f8564261b4b003157ec7c205 [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:47:19.429584 12279 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskToVWRe/build/release/bin/kudu as pid 23007
I20260806 03:47:19.429657 12279 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785988004637995-12279-0/minicluster-data/master-0/wal/instance
I20260806 03:47:19.429806 12279 external_mini_cluster.cc:1428] Setting key cfd9f8826c812f6f338bab7a729c0bdb
I20260806 03:47:19.430153 23119 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1faf2e28f8564261b4b003157ec7c205 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1faf2e28f8564261b4b003157ec7c205" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 38857 } }
I20260806 03:47:19.430267 23119 leader_election.cc:302] T 00000000000000000000000000000000 P 1faf2e28f8564261b4b003157ec7c205 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1faf2e28f8564261b4b003157ec7c205; no voters:
I20260806 03:47:19.430480 23119 leader_election.cc:288] T 00000000000000000000000000000000 P 1faf2e28f8564261b4b003157ec7c205 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 03:47:19.430540 23125 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1faf2e28f8564261b4b003157ec7c205 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 03:47:19.430719 23125 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1faf2e28f8564261b4b003157ec7c205 [term 1 LEADER]: Becoming Leader. State: Replica: 1faf2e28f8564261b4b003157ec7c205, State: Running, Role: LEADER
I20260806 03:47:19.430826 23119 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1faf2e28f8564261b4b003157ec7c205 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 03:47:19.430817 23125 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1faf2e28f8564261b4b003157ec7c205 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1faf2e28f8564261b4b003157ec7c205" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 38857 } }
I20260806 03:47:19.431226 23125 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1faf2e28f8564261b4b003157ec7c205 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1faf2e28f8564261b4b003157ec7c205. Latest consensus state: current_term: 1 leader_uuid: "1faf2e28f8564261b4b003157ec7c205" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1faf2e28f8564261b4b003157ec7c205" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 38857 } } }
I20260806 03:47:19.431241 23128 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1faf2e28f8564261b4b003157ec7c205 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1faf2e28f8564261b4b003157ec7c205" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1faf2e28f8564261b4b003157ec7c205" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 38857 } } }
I20260806 03:47:19.431318 23125 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1faf2e28f8564261b4b003157ec7c205 [sys.catalog]: This master's current role is: LEADER
I20260806 03:47:19.431320 23128 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1faf2e28f8564261b4b003157ec7c205 [sys.catalog]: This master's current role is: LEADER
I20260806 03:47:19.432224 23134 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 03:47:19.432775 23134 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 03:47:19.435071 23134 catalog_manager.cc:1347] Generated new cluster ID: 5b2d2df67aa84fd6bbb8287df73d615e
I20260806 03:47:19.435158 23134 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 03:47:19 dist-test-slave-6ntk krb5kdc[22990](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988039, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.11.253.254@KRBTEST.COM
I20260806 03:47:19.442414 23127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:19.430486 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56056 (local address 127.11.253.254:38857)
0806 03:47:19.430912 (+ 426us) server_negotiation.cc:206] Beginning negotiation
0806 03:47:19.430921 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 03:47:19.430958 (+ 37us) server_negotiation.cc:407] Connection header received
0806 03:47:19.431089 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:47:19.431095 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:47:19.431151 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:47:19.432206 (+ 1055us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:47:19.432333 (+ 127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:19.433863 (+ 1530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:19.434916 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:19.435538 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:19.435653 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:19.438663 (+ 3010us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:47:19.438667 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:47:19.438667 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:47:19.438679 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:47:19.440670 (+ 1991us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:47:19.441098 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:47:19.441101 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:47:19.441101 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:47:19.441211 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:47:19.441441 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:47:19.441442 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:47:19.441442 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:47:19.441738 (+ 296us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:47:19.441916 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0806 03:47:19.442060 (+ 144us) server_negotiation.cc:299] Negotiation successful
0806 03:47:19.442190 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":369,"thread_start_us":71,"threads_started":1}
I20260806 03:47:19.464449 23134 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 03:47:19.465011 23134 catalog_manager.cc:1504] Loading token signing keys...
I20260806 03:47:19.472046 23134 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1faf2e28f8564261b4b003157ec7c205: Generated new TSK 0
I20260806 03:47:19.472277 23134 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.11.253.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.11.253.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.11.253.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785988004637995-12279-0/krb5kdc/oryx_127.11.253.193.keytab.
Entry for principal oryx/127.11.253.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785988004637995-12279-0/krb5kdc/oryx_127.11.253.193.keytab.
I20260806 03:47:19.506101 12279 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskToVWRe/build/release/bin/kudu
/tmp/dist-test-taskToVWRe/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785988004637995-12279-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785988004637995-12279-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785988004637995-12279-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785988004637995-12279-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.253.193:0
--local_ip_for_outbound_sockets=127.11.253.193
--webserver_interface=127.11.253.193
--webserver_port=0
--tserver_master_addrs=127.11.253.254:38857
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785988004637995-12279-0/krb5kdc/oryx_127.11.253.193.keytab
--principal=oryx/127.11.253.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.253.212:35693
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785988004637995-12279-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785988004637995-12279-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785988004637995-12279-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:47:19.593858 23154 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:47:19.594094 23154 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:47:19.594120 23154 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:47:19.595194 23154 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:47:19.595235 23154 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:47:19.595255 23154 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 03:47:19.595273 23154 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.253.193
I20260806 03:47:19.596671 23154 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.253.212:35693
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785988004637995-12279-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785988004637995-12279-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785988004637995-12279-0/krb5kdc/oryx_127.11.253.193.keytab
--principal=oryx/127.11.253.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.253.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785988004637995-12279-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.253.193
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.11.253.254:38857
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23154
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.253.193
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785988004637995-12279-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:47:19.596869 23154 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:47:19.597038 23154 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:47:19.602327 23166 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:47:19.602442 23154 server_base.cc:1034] running on GCE node
W20260806 03:47:19.602336 23161 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:47:19.602337 23162 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:47:19.602800 23154 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:47:19.603019 23154 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:47:19.604207 23154 hybrid_clock.cc:648] HybridClock initialized: now 1785988039604195 us; error 35 us; skew 500 ppm
Aug 06 03:47:19 dist-test-slave-6ntk krb5kdc[22990](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988039, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.253.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:47:19.606935 23154 init.cc:377] Logged in from keytab as oryx/127.11.253.193@KRBTEST.COM (short username oryx)
I20260806 03:47:19.607326 23154 webserver.cc:460] Webserver started at http://127.11.253.193:39667/ using document root <none> and password file <none>
I20260806 03:47:19.607575 23154 fs_manager.cc:359] Metadata directory not provided
I20260806 03:47:19.607620 23154 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:47:19.607802 23154 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:47:19.608861 23154 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785988004637995-12279-0/minicluster-data/ts-0/data/instance:
uuid: "99f0ce1db669473cb520703bff465d2f"
format_stamp: "Formatted at 2026-08-06 03:47:19 on dist-test-slave-6ntk"
server_key: "059d86ad49f42c2303a9845e029bda92"
server_key_iv: "b79ee6bbdd753b6d4a5a9e912b5a508b"
server_key_version: "encryptionkey@0"
I20260806 03:47:19.609241 23154 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785988004637995-12279-0/minicluster-data/ts-0/wal/instance:
uuid: "99f0ce1db669473cb520703bff465d2f"
format_stamp: "Formatted at 2026-08-06 03:47:19 on dist-test-slave-6ntk"
server_key: "059d86ad49f42c2303a9845e029bda92"
server_key_iv: "b79ee6bbdd753b6d4a5a9e912b5a508b"
server_key_version: "encryptionkey@0"
I20260806 03:47:19.610672 23154 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 03:47:19.611603 23174 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:47:19.611824 23154 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 03:47:19.611935 23154 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785988004637995-12279-0/minicluster-data/ts-0/data,/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785988004637995-12279-0/minicluster-data/ts-0/wal
uuid: "99f0ce1db669473cb520703bff465d2f"
format_stamp: "Formatted at 2026-08-06 03:47:19 on dist-test-slave-6ntk"
server_key: "059d86ad49f42c2303a9845e029bda92"
server_key_iv: "b79ee6bbdd753b6d4a5a9e912b5a508b"
server_key_version: "encryptionkey@0"
I20260806 03:47:19.612015 23154 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785988004637995-12279-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785988004637995-12279-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785988004637995-12279-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:47:19.631069 23154 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:47:19.633805 23154 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:47:19.633951 23154 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:47:19.634565 23154 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 03:47:19.634610 23154 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 03:47:19.634634 23154 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 03:47:19.634647 23154 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 03:47:19 dist-test-slave-6ntk krb5kdc[22990](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988039, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.253.193@KRBTEST.COM for oryx/127.11.253.254@KRBTEST.COM
I20260806 03:47:19.642438 23154 rpc_server.cc:307] RPC server started. Bound to: 127.11.253.193:33047
I20260806 03:47:19.642540 23335 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.253.193:33047 every 8 connection(s)
I20260806 03:47:19.642859 23154 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785988004637995-12279-0/minicluster-data/ts-0/data/info.pb
I20260806 03:47:19.645781 23127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:19.635011 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.253.193:46335 (local address 127.11.253.254:38857)
0806 03:47:19.635082 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0806 03:47:19.635084 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:47:19.635446 (+ 362us) server_negotiation.cc:407] Connection header received
0806 03:47:19.635997 (+ 551us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:47:19.635998 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:47:19.636012 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:47:19.636096 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:47:19.637190 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:19.638073 (+ 883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:19.639246 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:19.639642 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:19.639734 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:19.642718 (+ 2984us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:47:19.642719 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:47:19.642720 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:47:19.642728 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:47:19.644407 (+ 1679us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:47:19.644832 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:47:19.644833 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:47:19.644833 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:47:19.644904 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:47:19.645152 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:47:19.645153 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:47:19.645153 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:47:19.645320 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:47:19.645386 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0806 03:47:19.645630 (+ 244us) server_negotiation.cc:299] Negotiation successful
0806 03:47:19.645709 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260806 03:47:19.646050 23183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:19.635009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:38857 (local address 127.11.253.193:46335)
0806 03:47:19.635242 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0806 03:47:19.635304 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0806 03:47:19.635881 (+ 577us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:47:19.636212 (+ 331us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:47:19.636218 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:47:19.636701 (+ 483us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:47:19.637060 (+ 359us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:19.637068 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:19.638192 (+ 1124us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:19.638193 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:19.639128 (+ 935us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:19.639131 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:19.639770 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:19.639771 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:19.639990 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:19.640583 (+ 593us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:47:19.640592 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:47:19.642585 (+ 1993us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:47:19.644494 (+ 1909us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:47:19.644500 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:47:19.644502 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:47:19.644733 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:47:19.644977 (+ 244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:47:19.644978 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:47:19.644978 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:47:19.645091 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:47:19.645384 (+ 293us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:47:19.645389 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:47:19.645507 (+ 118us) client_negotiation.cc:769] Sending connection context
0806 03:47:19.645594 (+ 87us) client_negotiation.cc:240] Negotiation successful
0806 03:47:19.645726 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":138,"threads_started":1}
I20260806 03:47:19.646787 23337 heartbeater.cc:347] Connected to a master server at 127.11.253.254:38857
I20260806 03:47:19.646878 23337 heartbeater.cc:464] Registering TS with master...
I20260806 03:47:19.647087 23337 heartbeater.cc:511] Master 127.11.253.254:38857 requested a full tablet report, sending...
I20260806 03:47:19.647647 23048 ts_manager.cc:194] Registered new tserver with Master: 99f0ce1db669473cb520703bff465d2f (127.11.253.193:33047)
I20260806 03:47:19.648433 23048 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.11.253.193@KRBTEST.COM'} at 127.11.253.193:46335
I20260806 03:47:19.652249 12279 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskToVWRe/build/release/bin/kudu as pid 23154
I20260806 03:47:19.652395 12279 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785988004637995-12279-0/minicluster-data/ts-0/wal/instance
I20260806 03:47:19.652637 12279 external_mini_cluster.cc:1428] Setting key 2fb7ac8763de06092983ae7428b1f0b8
I20260806 03:47:19.654379 23127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:19.647906 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.253.193:56695 (local address 127.11.253.254:38857)
0806 03:47:19.647969 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0806 03:47:19.647971 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:47:19.648821 (+ 850us) server_negotiation.cc:407] Connection header received
0806 03:47:19.648849 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:47:19.648850 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:47:19.648862 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:47:19.648943 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:47:19.649339 (+ 396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:19.649813 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:19.650533 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:19.650727 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:19.650784 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:19.651711 (+ 927us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:47:19.651713 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:47:19.651713 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:47:19.651720 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:47:19.653157 (+ 1437us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:47:19.653527 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:47:19.653528 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:47:19.653529 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:47:19.653586 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:47:19.653789 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:47:19.653790 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:47:19.653790 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:47:19.653957 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:47:19.654038 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0806 03:47:19.654196 (+ 158us) server_negotiation.cc:299] Negotiation successful
0806 03:47:19.654277 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260806 03:47:19.654389 23345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:19.647900 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:38857 (local address 127.11.253.193:56695)
0806 03:47:19.648723 (+ 823us) negotiation.cc:107] Waiting for socket to connect
0806 03:47:19.648736 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 03:47:19.648796 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:47:19.648967 (+ 171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:47:19.648967 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:47:19.649180 (+ 213us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:47:19.649253 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:19.649255 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:19.649962 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:19.649963 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:19.650458 (+ 495us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:19.650459 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:19.650797 (+ 338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:19.650797 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:19.650851 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:19.651342 (+ 491us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:47:19.651347 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:47:19.651633 (+ 286us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:47:19.653266 (+ 1633us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:47:19.653267 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:47:19.653268 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:47:19.653442 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:47:19.653653 (+ 211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:47:19.653654 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:47:19.653654 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:47:19.653722 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:47:19.654042 (+ 320us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:47:19.654043 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:47:19.654095 (+ 52us) client_negotiation.cc:769] Sending connection context
0806 03:47:19.654169 (+ 74us) client_negotiation.cc:240] Negotiation successful
0806 03:47:19.654277 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":788,"thread_start_us":70,"threads_started":1}
WARNING: no policy specified for oryx/127.11.253.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.11.253.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.11.253.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785988004637995-12279-0/krb5kdc/oryx_127.11.253.194.keytab.
Entry for principal oryx/127.11.253.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785988004637995-12279-0/krb5kdc/oryx_127.11.253.194.keytab.
I20260806 03:47:19.680541 12279 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskToVWRe/build/release/bin/kudu
/tmp/dist-test-taskToVWRe/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785988004637995-12279-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785988004637995-12279-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785988004637995-12279-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785988004637995-12279-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.253.194:0
--local_ip_for_outbound_sockets=127.11.253.194
--webserver_interface=127.11.253.194
--webserver_port=0
--tserver_master_addrs=127.11.253.254:38857
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785988004637995-12279-0/krb5kdc/oryx_127.11.253.194.keytab
--principal=oryx/127.11.253.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.253.212:35693
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785988004637995-12279-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785988004637995-12279-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785988004637995-12279-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:47:19.764631 23348 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:47:19.764860 23348 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:47:19.764884 23348 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:47:19.766098 23348 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:47:19.766144 23348 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:47:19.766165 23348 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 03:47:19.766213 23348 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.253.194
I20260806 03:47:19.767786 23348 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.253.212:35693
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785988004637995-12279-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785988004637995-12279-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785988004637995-12279-0/krb5kdc/oryx_127.11.253.194.keytab
--principal=oryx/127.11.253.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.253.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785988004637995-12279-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.253.194
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.11.253.254:38857
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23348
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.253.194
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785988004637995-12279-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:47:19.768057 23348 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:47:19.768294 23348 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:47:19.773521 23361 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:47:19.773555 23353 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:47:19.773526 23357 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:47:19.773622 23348 server_base.cc:1034] running on GCE node
I20260806 03:47:19.773973 23348 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:47:19.774235 23348 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:47:19.775408 23348 hybrid_clock.cc:648] HybridClock initialized: now 1785988039775422 us; error 54 us; skew 500 ppm
Aug 06 03:47:19 dist-test-slave-6ntk krb5kdc[22990](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988039, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.253.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:47:19.778017 23348 init.cc:377] Logged in from keytab as oryx/127.11.253.194@KRBTEST.COM (short username oryx)
I20260806 03:47:19.778520 23348 webserver.cc:460] Webserver started at http://127.11.253.194:33511/ using document root <none> and password file <none>
I20260806 03:47:19.778777 23348 fs_manager.cc:359] Metadata directory not provided
I20260806 03:47:19.778846 23348 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:47:19.778996 23348 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:47:19.780089 23348 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785988004637995-12279-0/minicluster-data/ts-1/data/instance:
uuid: "83f908250a6b4cdc9f27b1141221d4d9"
format_stamp: "Formatted at 2026-08-06 03:47:19 on dist-test-slave-6ntk"
server_key: "c06b8bc8247b1192b1e49a07ffce6615"
server_key_iv: "e4daf0497be213852624cfffa95babd6"
server_key_version: "encryptionkey@0"
I20260806 03:47:19.780473 23348 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785988004637995-12279-0/minicluster-data/ts-1/wal/instance:
uuid: "83f908250a6b4cdc9f27b1141221d4d9"
format_stamp: "Formatted at 2026-08-06 03:47:19 on dist-test-slave-6ntk"
server_key: "c06b8bc8247b1192b1e49a07ffce6615"
server_key_iv: "e4daf0497be213852624cfffa95babd6"
server_key_version: "encryptionkey@0"
I20260806 03:47:19.781832 23348 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 03:47:19.782610 23370 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:47:19.782785 23348 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 03:47:19.782892 23348 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785988004637995-12279-0/minicluster-data/ts-1/data,/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785988004637995-12279-0/minicluster-data/ts-1/wal
uuid: "83f908250a6b4cdc9f27b1141221d4d9"
format_stamp: "Formatted at 2026-08-06 03:47:19 on dist-test-slave-6ntk"
server_key: "c06b8bc8247b1192b1e49a07ffce6615"
server_key_iv: "e4daf0497be213852624cfffa95babd6"
server_key_version: "encryptionkey@0"
I20260806 03:47:19.782979 23348 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785988004637995-12279-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785988004637995-12279-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785988004637995-12279-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:47:19.798686 23348 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:47:19.800956 23348 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:47:19.801115 23348 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:47:19.801636 23348 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 03:47:19.801702 23348 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 03:47:19.801766 23348 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 03:47:19.801797 23348 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 03:47:19 dist-test-slave-6ntk krb5kdc[22990](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988039, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.253.194@KRBTEST.COM for oryx/127.11.253.254@KRBTEST.COM
I20260806 03:47:19.809162 23348 rpc_server.cc:307] RPC server started. Bound to: 127.11.253.194:34927
I20260806 03:47:19.809249 23534 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.253.194:34927 every 8 connection(s)
I20260806 03:47:19.809598 23348 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785988004637995-12279-0/minicluster-data/ts-1/data/info.pb
I20260806 03:47:19.811856 23127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:19.802220 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.253.194:38083 (local address 127.11.253.254:38857)
0806 03:47:19.802283 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0806 03:47:19.802286 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 03:47:19.802467 (+ 181us) server_negotiation.cc:407] Connection header received
0806 03:47:19.803042 (+ 575us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:47:19.803044 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:47:19.803058 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:47:19.803109 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:47:19.803870 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:19.804665 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:19.806229 (+ 1564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:19.806434 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:19.806520 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:19.809089 (+ 2569us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:47:19.809090 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:47:19.809090 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:47:19.809099 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:47:19.810707 (+ 1608us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:47:19.811106 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:47:19.811107 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:47:19.811107 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:47:19.811157 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:47:19.811333 (+ 176us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:47:19.811334 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:47:19.811334 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:47:19.811436 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:47:19.811485 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0806 03:47:19.811724 (+ 239us) server_negotiation.cc:299] Negotiation successful
0806 03:47:19.811784 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260806 03:47:19.812039 23381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:19.802197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:38857 (local address 127.11.253.194:38083)
0806 03:47:19.802350 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0806 03:47:19.802385 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0806 03:47:19.802922 (+ 537us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:47:19.803184 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:47:19.803188 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:47:19.803484 (+ 296us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:47:19.803759 (+ 275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:19.803765 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:19.804802 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:19.804804 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:19.806129 (+ 1325us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:19.806132 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:19.806552 (+ 420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:19.806553 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:19.806678 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:19.807214 (+ 536us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:47:19.807224 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:47:19.808950 (+ 1726us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:47:19.810784 (+ 1834us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:47:19.810790 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:47:19.810791 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:47:19.811036 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:47:19.811200 (+ 164us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:47:19.811200 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:47:19.811200 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:47:19.811285 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:47:19.811493 (+ 208us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:47:19.811499 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:47:19.811613 (+ 114us) client_negotiation.cc:769] Sending connection context
0806 03:47:19.811696 (+ 83us) client_negotiation.cc:240] Negotiation successful
0806 03:47:19.811816 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":76,"threads_started":1}
I20260806 03:47:19.812565 23535 heartbeater.cc:347] Connected to a master server at 127.11.253.254:38857
I20260806 03:47:19.812669 23535 heartbeater.cc:464] Registering TS with master...
I20260806 03:47:19.812855 23535 heartbeater.cc:511] Master 127.11.253.254:38857 requested a full tablet report, sending...
I20260806 03:47:19.813278 23048 ts_manager.cc:194] Registered new tserver with Master: 83f908250a6b4cdc9f27b1141221d4d9 (127.11.253.194:34927)
I20260806 03:47:19.813696 23048 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.11.253.194@KRBTEST.COM'} at 127.11.253.194:38083
I20260806 03:47:19.816094 12279 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskToVWRe/build/release/bin/kudu as pid 23348
I20260806 03:47:19.816234 12279 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785988004637995-12279-0/minicluster-data/ts-1/wal/instance
I20260806 03:47:19.816494 12279 external_mini_cluster.cc:1428] Setting key ea41a1e20e513bb89bceb02dd5e44c3f
I20260806 03:47:19.819501 23542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:19.813634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:38857 (local address 127.11.253.194:37405)
0806 03:47:19.813749 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0806 03:47:19.813761 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 03:47:19.813840 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:47:19.814026 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:47:19.814027 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:47:19.814203 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:47:19.814251 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:19.814252 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:19.814944 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:19.814945 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:19.815542 (+ 597us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:19.815545 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:19.815855 (+ 310us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:19.815856 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:19.815936 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:19.816435 (+ 499us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:47:19.816442 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:47:19.816721 (+ 279us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:47:19.818382 (+ 1661us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:47:19.818383 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:47:19.818383 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:47:19.818620 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:47:19.818808 (+ 188us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:47:19.818809 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:47:19.818809 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:47:19.818885 (+ 76us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:47:19.819249 (+ 364us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:47:19.819250 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:47:19.819288 (+ 38us) client_negotiation.cc:769] Sending connection context
0806 03:47:19.819350 (+ 62us) client_negotiation.cc:240] Negotiation successful
0806 03:47:19.819429 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":91,"thread_start_us":66,"threads_started":1}
I20260806 03:47:19.819598 23127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:19.813697 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.253.194:37405 (local address 127.11.253.254:38857)
0806 03:47:19.813743 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0806 03:47:19.813745 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:47:19.813814 (+ 69us) server_negotiation.cc:407] Connection header received
0806 03:47:19.813936 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:47:19.813937 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:47:19.813948 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:47:19.814013 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:47:19.814332 (+ 319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:19.814861 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:19.815613 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:19.815789 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:19.815843 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:19.816830 (+ 987us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:47:19.816831 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:47:19.816831 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:47:19.816843 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:47:19.818299 (+ 1456us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:47:19.818695 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:47:19.818695 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:47:19.818696 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:47:19.818744 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:47:19.818967 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:47:19.818968 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:47:19.818968 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:47:19.819179 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:47:19.819254 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0806 03:47:19.819398 (+ 144us) server_negotiation.cc:299] Negotiation successful
0806 03:47:19.819481 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
WARNING: no policy specified for oryx/127.11.253.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.11.253.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.11.253.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785988004637995-12279-0/krb5kdc/oryx_127.11.253.195.keytab.
Entry for principal oryx/127.11.253.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785988004637995-12279-0/krb5kdc/oryx_127.11.253.195.keytab.
I20260806 03:47:19.845036 12279 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskToVWRe/build/release/bin/kudu
/tmp/dist-test-taskToVWRe/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785988004637995-12279-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785988004637995-12279-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785988004637995-12279-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785988004637995-12279-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.253.195:0
--local_ip_for_outbound_sockets=127.11.253.195
--webserver_interface=127.11.253.195
--webserver_port=0
--tserver_master_addrs=127.11.253.254:38857
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785988004637995-12279-0/krb5kdc/oryx_127.11.253.195.keytab
--principal=oryx/127.11.253.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.253.212:35693
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785988004637995-12279-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785988004637995-12279-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785988004637995-12279-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:47:19.929082 23545 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:47:19.929323 23545 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:47:19.929361 23545 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:47:19.930546 23545 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:47:19.930619 23545 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:47:19.930652 23545 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 03:47:19.930694 23545 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.253.195
I20260806 03:47:19.932395 23545 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.253.212:35693
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785988004637995-12279-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785988004637995-12279-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785988004637995-12279-0/krb5kdc/oryx_127.11.253.195.keytab
--principal=oryx/127.11.253.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.253.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785988004637995-12279-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.253.195
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.11.253.254:38857
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23545
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.253.195
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785988004637995-12279-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:47:19.932663 23545 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:47:19.932919 23545 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:47:19.937985 23555 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:47:19.937985 23557 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:47:19.938040 23545 server_base.cc:1034] running on GCE node
W20260806 03:47:19.938024 23553 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:47:19.938419 23545 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:47:19.938711 23545 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:47:19.939877 23545 hybrid_clock.cc:648] HybridClock initialized: now 1785988039939875 us; error 40 us; skew 500 ppm
Aug 06 03:47:19 dist-test-slave-6ntk krb5kdc[22990](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988039, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.253.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:47:19.942698 23545 init.cc:377] Logged in from keytab as oryx/127.11.253.195@KRBTEST.COM (short username oryx)
I20260806 03:47:19.943192 23545 webserver.cc:460] Webserver started at http://127.11.253.195:41245/ using document root <none> and password file <none>
I20260806 03:47:19.943472 23545 fs_manager.cc:359] Metadata directory not provided
I20260806 03:47:19.943547 23545 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:47:19.943698 23545 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:47:19.944725 23545 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785988004637995-12279-0/minicluster-data/ts-2/data/instance:
uuid: "997d20698f9b493eb729667604016e89"
format_stamp: "Formatted at 2026-08-06 03:47:19 on dist-test-slave-6ntk"
server_key: "039d9afdc20d130507998534c2fec7e9"
server_key_iv: "0da2dbe6d778792435221cf438c7f57d"
server_key_version: "encryptionkey@0"
I20260806 03:47:19.945099 23545 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785988004637995-12279-0/minicluster-data/ts-2/wal/instance:
uuid: "997d20698f9b493eb729667604016e89"
format_stamp: "Formatted at 2026-08-06 03:47:19 on dist-test-slave-6ntk"
server_key: "039d9afdc20d130507998534c2fec7e9"
server_key_iv: "0da2dbe6d778792435221cf438c7f57d"
server_key_version: "encryptionkey@0"
I20260806 03:47:19.946549 23545 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 03:47:19.947374 23567 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:47:19.947551 23545 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 03:47:19.947660 23545 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785988004637995-12279-0/minicluster-data/ts-2/data,/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785988004637995-12279-0/minicluster-data/ts-2/wal
uuid: "997d20698f9b493eb729667604016e89"
format_stamp: "Formatted at 2026-08-06 03:47:19 on dist-test-slave-6ntk"
server_key: "039d9afdc20d130507998534c2fec7e9"
server_key_iv: "0da2dbe6d778792435221cf438c7f57d"
server_key_version: "encryptionkey@0"
I20260806 03:47:19.947734 23545 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785988004637995-12279-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785988004637995-12279-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785988004637995-12279-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:47:19.968375 23545 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:47:19.970947 23545 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:47:19.971117 23545 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:47:19.971674 23545 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 03:47:19.971741 23545 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 03:47:19.971786 23545 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 03:47:19.971813 23545 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 03:47:19.978818 23545 rpc_server.cc:307] RPC server started. Bound to: 127.11.253.195:37749
Aug 06 03:47:19 dist-test-slave-6ntk krb5kdc[22990](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988039, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.253.195@KRBTEST.COM for oryx/127.11.253.254@KRBTEST.COM
I20260806 03:47:19.978965 23734 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.253.195:37749 every 8 connection(s)
I20260806 03:47:19.979220 23545 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785988004637995-12279-0/minicluster-data/ts-2/data/info.pb
I20260806 03:47:19.980769 12279 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskToVWRe/build/release/bin/kudu as pid 23545
I20260806 03:47:19.980847 12279 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785988004637995-12279-0/minicluster-data/ts-2/wal/instance
I20260806 03:47:19.981042 12279 external_mini_cluster.cc:1428] Setting key 29b7b0d7e827392f2db3af1ee8d4edc3
I20260806 03:47:19.982527 23127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:19.972185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.253.195:55365 (local address 127.11.253.254:38857)
0806 03:47:19.972242 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0806 03:47:19.972245 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 03:47:19.972496 (+ 251us) server_negotiation.cc:407] Connection header received
0806 03:47:19.973063 (+ 567us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:47:19.973065 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:47:19.973080 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:47:19.973171 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:47:19.974070 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:19.974586 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:19.976185 (+ 1599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:19.976553 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:19.976634 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:19.979467 (+ 2833us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:47:19.979468 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:47:19.979469 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:47:19.979478 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:47:19.981059 (+ 1581us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:47:19.981663 (+ 604us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:47:19.981664 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:47:19.981665 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:47:19.981735 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:47:19.982027 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:47:19.982028 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:47:19.982028 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:47:19.982153 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:47:19.982211 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0806 03:47:19.982405 (+ 194us) server_negotiation.cc:299] Negotiation successful
0806 03:47:19.982465 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260806 03:47:19.982736 23580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:19.972180 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:38857 (local address 127.11.253.195:55365)
0806 03:47:19.972355 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0806 03:47:19.972397 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 03:47:19.972956 (+ 559us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:47:19.973225 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:47:19.973229 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:47:19.973638 (+ 409us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:47:19.973962 (+ 324us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:19.973968 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:19.974683 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:19.974684 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:19.976087 (+ 1403us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:19.976090 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:19.976677 (+ 587us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:19.976678 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:19.976828 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:19.977357 (+ 529us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:47:19.977368 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:47:19.979352 (+ 1984us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:47:19.981169 (+ 1817us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:47:19.981183 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:47:19.981185 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:47:19.981554 (+ 369us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:47:19.981819 (+ 265us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:47:19.981820 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:47:19.981820 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:47:19.981978 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:47:19.982203 (+ 225us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:47:19.982206 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:47:19.982315 (+ 109us) client_negotiation.cc:769] Sending connection context
0806 03:47:19.982382 (+ 67us) client_negotiation.cc:240] Negotiation successful
0806 03:47:19.982498 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":78,"threads_started":1}
I20260806 03:47:19.983323 23735 heartbeater.cc:347] Connected to a master server at 127.11.253.254:38857
I20260806 03:47:19.983421 23735 heartbeater.cc:464] Registering TS with master...
I20260806 03:47:19.983623 23735 heartbeater.cc:511] Master 127.11.253.254:38857 requested a full tablet report, sending...
I20260806 03:47:19.984069 23048 ts_manager.cc:194] Registered new tserver with Master: 997d20698f9b493eb729667604016e89 (127.11.253.195:37749)
I20260806 03:47:19.984584 23048 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.11.253.195@KRBTEST.COM'} at 127.11.253.195:55365
I20260806 03:47:19.989851 23127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:19.984399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.253.195:53585 (local address 127.11.253.254:38857)
0806 03:47:19.984443 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0806 03:47:19.984445 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:47:19.984552 (+ 107us) server_negotiation.cc:407] Connection header received
0806 03:47:19.984623 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:47:19.984623 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:47:19.984631 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:47:19.984701 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:47:19.985021 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:19.985545 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:19.986333 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:19.986585 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:19.986667 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:19.987537 (+ 870us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:47:19.987538 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:47:19.987538 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:47:19.987546 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:47:19.988686 (+ 1140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:47:19.989021 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:47:19.989022 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:47:19.989023 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:47:19.989076 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:47:19.989278 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:47:19.989279 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:47:19.989279 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:47:19.989450 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:47:19.989527 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0806 03:47:19.989694 (+ 167us) server_negotiation.cc:299] Negotiation successful
0806 03:47:19.989761 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260806 03:47:19.989902 23742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:19.984333 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:38857 (local address 127.11.253.195:53585)
0806 03:47:19.984482 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0806 03:47:19.984495 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 03:47:19.984564 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:47:19.984707 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:47:19.984708 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:47:19.984889 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:47:19.984942 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:19.984943 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:19.985635 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:19.985636 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:19.986249 (+ 613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:19.986251 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:19.986670 (+ 419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:19.986671 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:19.986742 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:19.987181 (+ 439us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:47:19.987185 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:47:19.987449 (+ 264us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:47:19.988771 (+ 1322us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:47:19.988772 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:47:19.988773 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:47:19.988948 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:47:19.989152 (+ 204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:47:19.989153 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:47:19.989153 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:47:19.989213 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:47:19.989531 (+ 318us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:47:19.989532 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:47:19.989578 (+ 46us) client_negotiation.cc:769] Sending connection context
0806 03:47:19.989655 (+ 77us) client_negotiation.cc:240] Negotiation successful
0806 03:47:19.989759 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":85,"threads_started":1}
I20260806 03:47:19.992766 12279 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 06 03:47:20 dist-test-slave-6ntk krb5kdc[22990](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988040, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 06 03:47:20 dist-test-slave-6ntk krb5kdc[22990](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.11.253.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 03:47:20 dist-test-slave-6ntk krb5kdc[22990](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.11.253.254@KRBTEST.COM, Server not found in Kerberos database
W20260806 03:47:20.016278 23127 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 03:47:20.010138 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56064 (local address 127.11.253.254:38857)
0806 03:47:20.010212 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0806 03:47:20.010214 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:47:20.010221 (+ 7us) server_negotiation.cc:407] Connection header received
0806 03:47:20.010245 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:47:20.010246 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:47:20.010255 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:47:20.010331 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:47:20.010921 (+ 590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:20.011416 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:20.012171 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:20.012370 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:20.012468 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:20.016171 (+ 3703us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56064: BlockingRecv error: recv got EOF from 127.0.0.1:56064 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Aug 06 03:47:20 dist-test-slave-6ntk krb5kdc[22990](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988040, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.11.253.254@KRBTEST.COM
I20260806 03:47:20.023977 23127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:20.017068 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56072 (local address 127.11.253.254:38857)
0806 03:47:20.017126 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0806 03:47:20.017128 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:47:20.017221 (+ 93us) server_negotiation.cc:407] Connection header received
0806 03:47:20.017274 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:47:20.017274 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:47:20.017284 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:47:20.017337 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:47:20.017822 (+ 485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:20.018374 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:20.019054 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:20.019228 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:20.019278 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:20.021345 (+ 2067us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:47:20.021346 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:47:20.021347 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:47:20.021355 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:47:20.022841 (+ 1486us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:47:20.023203 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:47:20.023204 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:47:20.023204 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:47:20.023254 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:47:20.023479 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:47:20.023480 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:47:20.023480 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:47:20.023613 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:47:20.023722 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0806 03:47:20.023826 (+ 104us) server_negotiation.cc:299] Negotiation successful
0806 03:47:20.023902 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260806 03:47:20.025938 23048 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56072:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 03:47:20.027516 23048 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 03:47:20.034770 23772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:20.030301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40468 (local address 127.11.253.193:33047)
0806 03:47:20.030700 (+ 399us) server_negotiation.cc:206] Beginning negotiation
0806 03:47:20.030705 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 03:47:20.030794 (+ 89us) server_negotiation.cc:407] Connection header received
0806 03:47:20.031208 (+ 414us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:47:20.031212 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:47:20.031229 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:47:20.031305 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 03:47:20.031490 (+ 185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:20.032294 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:20.033608 (+ 1314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:20.034321 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:20.034417 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:20.034473 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0806 03:47:20.034619 (+ 146us) server_negotiation.cc:299] Negotiation successful
0806 03:47:20.034660 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":358,"thread_start_us":96,"threads_started":1}
I20260806 03:47:20.034763 23773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:20.030154 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.193:33047 (local address 127.0.0.1:40468)
0806 03:47:20.030585 (+ 431us) negotiation.cc:107] Waiting for socket to connect
0806 03:47:20.030712 (+ 127us) client_negotiation.cc:174] Beginning negotiation
0806 03:47:20.030793 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:47:20.031380 (+ 587us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:47:20.031380 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:47:20.031382 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 03:47:20.031415 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:20.031416 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:20.032383 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:20.032384 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:20.033508 (+ 1124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:20.033512 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:20.034415 (+ 903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:20.034416 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:20.034495 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:20.034510 (+ 15us) client_negotiation.cc:769] Sending connection context
0806 03:47:20.034592 (+ 82us) client_negotiation.cc:240] Negotiation successful
0806 03:47:20.034652 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":394,"mutex_wait_us":81,"thread_start_us":88,"threads_started":1}
I20260806 03:47:20.035058 23767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:20.030432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.194:34927 (local address 127.0.0.1:57708)
0806 03:47:20.031031 (+ 599us) negotiation.cc:107] Waiting for socket to connect
0806 03:47:20.031040 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 03:47:20.031080 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:47:20.031347 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:47:20.031347 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:47:20.031349 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 03:47:20.031387 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:20.031388 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:20.032634 (+ 1246us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:20.032635 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:20.033965 (+ 1330us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:20.033967 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:20.034831 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:20.034832 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:20.034895 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:20.034901 (+ 6us) client_negotiation.cc:769] Sending connection context
0806 03:47:20.034955 (+ 54us) client_negotiation.cc:240] Negotiation successful
0806 03:47:20.035002 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":588,"mutex_wait_us":52,"thread_start_us":92,"threads_started":1}
I20260806 03:47:20.035329 23775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:20.030567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57708 (local address 127.11.253.194:34927)
0806 03:47:20.031175 (+ 608us) server_negotiation.cc:206] Beginning negotiation
0806 03:47:20.031187 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 03:47:20.031194 (+ 7us) server_negotiation.cc:407] Connection header received
0806 03:47:20.031225 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:47:20.031227 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:47:20.031262 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:47:20.031338 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 03:47:20.031443 (+ 105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:20.032554 (+ 1111us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:20.034060 (+ 1506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:20.034667 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:20.034745 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:20.034794 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0806 03:47:20.035185 (+ 391us) server_negotiation.cc:299] Negotiation successful
0806 03:47:20.035230 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":577,"thread_start_us":103,"threads_started":1}
I20260806 03:47:20.035759 23771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:20.030154 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.195:37749 (local address 127.0.0.1:43088)
0806 03:47:20.030649 (+ 495us) negotiation.cc:107] Waiting for socket to connect
0806 03:47:20.030833 (+ 184us) client_negotiation.cc:174] Beginning negotiation
0806 03:47:20.030887 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:47:20.031106 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:47:20.031110 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:47:20.031112 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 03:47:20.031187 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:20.031191 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:20.032594 (+ 1403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:20.032595 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:20.033917 (+ 1322us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:20.033921 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:20.035551 (+ 1630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:20.035552 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:20.035626 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:20.035630 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 03:47:20.035667 (+ 37us) client_negotiation.cc:240] Negotiation successful
0806 03:47:20.035703 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":478,"mutex_wait_us":39,"thread_start_us":70,"threads_started":1}
I20260806 03:47:20.035706 23241 tablet_service.cc:1467] Processing CreateTablet for tablet f898c8c90ece4ce6bf460e8ef402d407 (DEFAULT_TABLE table=test-table [id=4c200faa76214c5eaa1122547c47d95c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 03:47:20.035908 23438 tablet_service.cc:1467] Processing CreateTablet for tablet f898c8c90ece4ce6bf460e8ef402d407 (DEFAULT_TABLE table=test-table [id=4c200faa76214c5eaa1122547c47d95c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 03:47:20.036147 23241 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f898c8c90ece4ce6bf460e8ef402d407. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:47:20.036172 23774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:20.030457 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43088 (local address 127.11.253.195:37749)
0806 03:47:20.030845 (+ 388us) server_negotiation.cc:206] Beginning negotiation
0806 03:47:20.030849 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 03:47:20.030889 (+ 40us) server_negotiation.cc:407] Connection header received
0806 03:47:20.030982 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:47:20.030984 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:47:20.031003 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:47:20.031098 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 03:47:20.031298 (+ 200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:20.032496 (+ 1198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:20.034032 (+ 1536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:20.034789 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:20.035329 (+ 540us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:20.035965 (+ 636us) server_negotiation.cc:1035] Waiting for connection context
0806 03:47:20.035994 (+ 29us) server_negotiation.cc:299] Negotiation successful
0806 03:47:20.036047 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":344,"thread_start_us":109,"threads_started":1}
I20260806 03:47:20.036212 23438 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f898c8c90ece4ce6bf460e8ef402d407. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:47:20.036711 23642 tablet_service.cc:1467] Processing CreateTablet for tablet f898c8c90ece4ce6bf460e8ef402d407 (DEFAULT_TABLE table=test-table [id=4c200faa76214c5eaa1122547c47d95c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 03:47:20.037050 23642 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f898c8c90ece4ce6bf460e8ef402d407. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:47:20.040549 23777 tablet_bootstrap.cc:492] T f898c8c90ece4ce6bf460e8ef402d407 P 997d20698f9b493eb729667604016e89: Bootstrap starting.
I20260806 03:47:20.041169 23776 tablet_bootstrap.cc:492] T f898c8c90ece4ce6bf460e8ef402d407 P 99f0ce1db669473cb520703bff465d2f: Bootstrap starting.
I20260806 03:47:20.041283 23777 tablet_bootstrap.cc:654] T f898c8c90ece4ce6bf460e8ef402d407 P 997d20698f9b493eb729667604016e89: Neither blocks nor log segments found. Creating new log.
I20260806 03:47:20.041947 23777 log.cc:824] T f898c8c90ece4ce6bf460e8ef402d407 P 997d20698f9b493eb729667604016e89: Log is configured to *not* fsync() on all Append() calls
I20260806 03:47:20.042481 23776 tablet_bootstrap.cc:654] T f898c8c90ece4ce6bf460e8ef402d407 P 99f0ce1db669473cb520703bff465d2f: Neither blocks nor log segments found. Creating new log.
I20260806 03:47:20.042832 23776 log.cc:824] T f898c8c90ece4ce6bf460e8ef402d407 P 99f0ce1db669473cb520703bff465d2f: Log is configured to *not* fsync() on all Append() calls
I20260806 03:47:20.042884 23777 tablet_bootstrap.cc:492] T f898c8c90ece4ce6bf460e8ef402d407 P 997d20698f9b493eb729667604016e89: No bootstrap required, opened a new log
I20260806 03:47:20.042948 23777 ts_tablet_manager.cc:1397] T f898c8c90ece4ce6bf460e8ef402d407 P 997d20698f9b493eb729667604016e89: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260806 03:47:20.042930 23779 tablet_bootstrap.cc:492] T f898c8c90ece4ce6bf460e8ef402d407 P 83f908250a6b4cdc9f27b1141221d4d9: Bootstrap starting.
I20260806 03:47:20.043519 23779 tablet_bootstrap.cc:654] T f898c8c90ece4ce6bf460e8ef402d407 P 83f908250a6b4cdc9f27b1141221d4d9: Neither blocks nor log segments found. Creating new log.
I20260806 03:47:20.043659 23776 tablet_bootstrap.cc:492] T f898c8c90ece4ce6bf460e8ef402d407 P 99f0ce1db669473cb520703bff465d2f: No bootstrap required, opened a new log
I20260806 03:47:20.043735 23776 ts_tablet_manager.cc:1397] T f898c8c90ece4ce6bf460e8ef402d407 P 99f0ce1db669473cb520703bff465d2f: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 03:47:20.043864 23779 log.cc:824] T f898c8c90ece4ce6bf460e8ef402d407 P 83f908250a6b4cdc9f27b1141221d4d9: Log is configured to *not* fsync() on all Append() calls
I20260806 03:47:20.044673 23779 tablet_bootstrap.cc:492] T f898c8c90ece4ce6bf460e8ef402d407 P 83f908250a6b4cdc9f27b1141221d4d9: No bootstrap required, opened a new log
I20260806 03:47:20.044746 23779 ts_tablet_manager.cc:1397] T f898c8c90ece4ce6bf460e8ef402d407 P 83f908250a6b4cdc9f27b1141221d4d9: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 03:47:20.044932 23777 raft_consensus.cc:348] T f898c8c90ece4ce6bf460e8ef402d407 P 997d20698f9b493eb729667604016e89 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997d20698f9b493eb729667604016e89" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 37749 } } peers { permanent_uuid: "99f0ce1db669473cb520703bff465d2f" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 33047 } } peers { permanent_uuid: "83f908250a6b4cdc9f27b1141221d4d9" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 34927 } }
I20260806 03:47:20.045087 23777 raft_consensus.cc:374] T f898c8c90ece4ce6bf460e8ef402d407 P 997d20698f9b493eb729667604016e89 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:47:20.045130 23777 raft_consensus.cc:729] T f898c8c90ece4ce6bf460e8ef402d407 P 997d20698f9b493eb729667604016e89 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 997d20698f9b493eb729667604016e89, State: Initialized, Role: FOLLOWER
I20260806 03:47:20.045261 23777 consensus_queue.cc:260] T f898c8c90ece4ce6bf460e8ef402d407 P 997d20698f9b493eb729667604016e89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997d20698f9b493eb729667604016e89" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 37749 } } peers { permanent_uuid: "99f0ce1db669473cb520703bff465d2f" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 33047 } } peers { permanent_uuid: "83f908250a6b4cdc9f27b1141221d4d9" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 34927 } }
I20260806 03:47:20.045535 23777 ts_tablet_manager.cc:1428] T f898c8c90ece4ce6bf460e8ef402d407 P 997d20698f9b493eb729667604016e89: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260806 03:47:20.045668 23735 heartbeater.cc:503] Master 127.11.253.254:38857 was elected leader, sending a full tablet report...
I20260806 03:47:20.045811 23776 raft_consensus.cc:348] T f898c8c90ece4ce6bf460e8ef402d407 P 99f0ce1db669473cb520703bff465d2f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997d20698f9b493eb729667604016e89" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 37749 } } peers { permanent_uuid: "99f0ce1db669473cb520703bff465d2f" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 33047 } } peers { permanent_uuid: "83f908250a6b4cdc9f27b1141221d4d9" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 34927 } }
I20260806 03:47:20.046526 23776 raft_consensus.cc:374] T f898c8c90ece4ce6bf460e8ef402d407 P 99f0ce1db669473cb520703bff465d2f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:47:20.046597 23776 raft_consensus.cc:729] T f898c8c90ece4ce6bf460e8ef402d407 P 99f0ce1db669473cb520703bff465d2f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99f0ce1db669473cb520703bff465d2f, State: Initialized, Role: FOLLOWER
I20260806 03:47:20.046736 23776 consensus_queue.cc:260] T f898c8c90ece4ce6bf460e8ef402d407 P 99f0ce1db669473cb520703bff465d2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997d20698f9b493eb729667604016e89" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 37749 } } peers { permanent_uuid: "99f0ce1db669473cb520703bff465d2f" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 33047 } } peers { permanent_uuid: "83f908250a6b4cdc9f27b1141221d4d9" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 34927 } }
I20260806 03:47:20.047031 23776 ts_tablet_manager.cc:1428] T f898c8c90ece4ce6bf460e8ef402d407 P 99f0ce1db669473cb520703bff465d2f: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 03:47:20.046895 23779 raft_consensus.cc:348] T f898c8c90ece4ce6bf460e8ef402d407 P 83f908250a6b4cdc9f27b1141221d4d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997d20698f9b493eb729667604016e89" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 37749 } } peers { permanent_uuid: "99f0ce1db669473cb520703bff465d2f" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 33047 } } peers { permanent_uuid: "83f908250a6b4cdc9f27b1141221d4d9" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 34927 } }
I20260806 03:47:20.047113 23779 raft_consensus.cc:374] T f898c8c90ece4ce6bf460e8ef402d407 P 83f908250a6b4cdc9f27b1141221d4d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:47:20.047057 23337 heartbeater.cc:503] Master 127.11.253.254:38857 was elected leader, sending a full tablet report...
I20260806 03:47:20.047164 23779 raft_consensus.cc:729] T f898c8c90ece4ce6bf460e8ef402d407 P 83f908250a6b4cdc9f27b1141221d4d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83f908250a6b4cdc9f27b1141221d4d9, State: Initialized, Role: FOLLOWER
I20260806 03:47:20.047317 23779 consensus_queue.cc:260] T f898c8c90ece4ce6bf460e8ef402d407 P 83f908250a6b4cdc9f27b1141221d4d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997d20698f9b493eb729667604016e89" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 37749 } } peers { permanent_uuid: "99f0ce1db669473cb520703bff465d2f" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 33047 } } peers { permanent_uuid: "83f908250a6b4cdc9f27b1141221d4d9" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 34927 } }
I20260806 03:47:20.047590 23779 ts_tablet_manager.cc:1428] T f898c8c90ece4ce6bf460e8ef402d407 P 83f908250a6b4cdc9f27b1141221d4d9: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 03:47:20.047735 23535 heartbeater.cc:503] Master 127.11.253.254:38857 was elected leader, sending a full tablet report...
W20260806 03:47:20.060220 23536 tablet.cc:2369] T f898c8c90ece4ce6bf460e8ef402d407 P 83f908250a6b4cdc9f27b1141221d4d9: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 03:47:20.143718 23338 tablet.cc:2369] T f898c8c90ece4ce6bf460e8ef402d407 P 99f0ce1db669473cb520703bff465d2f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 03:47:20.227932 23787 raft_consensus.cc:482] T f898c8c90ece4ce6bf460e8ef402d407 P 99f0ce1db669473cb520703bff465d2f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 03:47:20.228108 23787 raft_consensus.cc:504] T f898c8c90ece4ce6bf460e8ef402d407 P 99f0ce1db669473cb520703bff465d2f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997d20698f9b493eb729667604016e89" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 37749 } } peers { permanent_uuid: "99f0ce1db669473cb520703bff465d2f" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 33047 } } peers { permanent_uuid: "83f908250a6b4cdc9f27b1141221d4d9" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 34927 } }
I20260806 03:47:20.228598 23787 leader_election.cc:288] T f898c8c90ece4ce6bf460e8ef402d407 P 99f0ce1db669473cb520703bff465d2f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 997d20698f9b493eb729667604016e89 (127.11.253.195:37749), 83f908250a6b4cdc9f27b1141221d4d9 (127.11.253.194:34927)
W20260806 03:47:20.229861 23736 tablet.cc:2369] T f898c8c90ece4ce6bf460e8ef402d407 P 997d20698f9b493eb729667604016e89: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 03:47:20.232095 23774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:20.228965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.253.193:57845 (local address 127.11.253.195:37749)
0806 03:47:20.229026 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0806 03:47:20.229028 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:47:20.229039 (+ 11us) server_negotiation.cc:407] Connection header received
0806 03:47:20.229069 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:47:20.229070 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:47:20.229080 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:47:20.229164 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 03:47:20.229494 (+ 330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:20.230149 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:20.231210 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:20.231735 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:20.231809 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:20.231820 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0806 03:47:20.231981 (+ 161us) server_negotiation.cc:299] Negotiation successful
0806 03:47:20.232019 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260806 03:47:20.232116 23789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:20.228784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.195:37749 (local address 127.11.253.193:57845)
0806 03:47:20.228915 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0806 03:47:20.228928 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 03:47:20.228991 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:47:20.229336 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:47:20.229337 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:47:20.229339 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 03:47:20.229409 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:20.229410 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:20.230239 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:20.230240 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:20.231109 (+ 869us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:20.231112 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:20.231812 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:20.231813 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:20.231899 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:20.231903 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 03:47:20.231974 (+ 71us) client_negotiation.cc:240] Negotiation successful
0806 03:47:20.232019 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"mutex_wait_us":1,"thread_start_us":82,"threads_started":1}
I20260806 03:47:20.232225 23775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:20.229098 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.253.193:53429 (local address 127.11.253.194:34927)
0806 03:47:20.229174 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0806 03:47:20.229176 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:47:20.229191 (+ 15us) server_negotiation.cc:407] Connection header received
0806 03:47:20.229218 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:47:20.229219 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:47:20.229227 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:47:20.229287 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 03:47:20.229496 (+ 209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:20.230120 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:20.231357 (+ 1237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:20.231846 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:20.231932 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:20.231941 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0806 03:47:20.232104 (+ 163us) server_negotiation.cc:299] Negotiation successful
0806 03:47:20.232148 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260806 03:47:20.232254 23793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:20.228904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.194:34927 (local address 127.11.253.193:53429)
0806 03:47:20.229038 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0806 03:47:20.229059 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 03:47:20.229129 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:47:20.229339 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:47:20.229340 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:47:20.229342 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 03:47:20.229403 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:20.229405 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:20.230203 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:20.230204 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:20.231258 (+ 1054us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:20.231261 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:20.231931 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:20.231932 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:20.232007 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:20.232012 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 03:47:20.232069 (+ 57us) client_negotiation.cc:240] Negotiation successful
0806 03:47:20.232120 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"mutex_wait_us":45,"thread_start_us":62,"threads_started":1}
I20260806 03:47:20.232656 23472 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f898c8c90ece4ce6bf460e8ef402d407" candidate_uuid: "99f0ce1db669473cb520703bff465d2f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f908250a6b4cdc9f27b1141221d4d9" is_pre_election: true
I20260806 03:47:20.232848 23472 raft_consensus.cc:2454] T f898c8c90ece4ce6bf460e8ef402d407 P 83f908250a6b4cdc9f27b1141221d4d9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99f0ce1db669473cb520703bff465d2f in term 0.
I20260806 03:47:20.233136 23177 leader_election.cc:302] T f898c8c90ece4ce6bf460e8ef402d407 P 99f0ce1db669473cb520703bff465d2f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f908250a6b4cdc9f27b1141221d4d9, 99f0ce1db669473cb520703bff465d2f; no voters:
I20260806 03:47:20.233165 23675 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f898c8c90ece4ce6bf460e8ef402d407" candidate_uuid: "99f0ce1db669473cb520703bff465d2f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "997d20698f9b493eb729667604016e89" is_pre_election: true
I20260806 03:47:20.233292 23787 raft_consensus.cc:2781] T f898c8c90ece4ce6bf460e8ef402d407 P 99f0ce1db669473cb520703bff465d2f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 03:47:20.233310 23675 raft_consensus.cc:2454] T f898c8c90ece4ce6bf460e8ef402d407 P 997d20698f9b493eb729667604016e89 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99f0ce1db669473cb520703bff465d2f in term 0.
I20260806 03:47:20.233352 23787 raft_consensus.cc:482] T f898c8c90ece4ce6bf460e8ef402d407 P 99f0ce1db669473cb520703bff465d2f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 03:47:20.233386 23787 raft_consensus.cc:3037] T f898c8c90ece4ce6bf460e8ef402d407 P 99f0ce1db669473cb520703bff465d2f [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:47:20.234251 23787 raft_consensus.cc:504] T f898c8c90ece4ce6bf460e8ef402d407 P 99f0ce1db669473cb520703bff465d2f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997d20698f9b493eb729667604016e89" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 37749 } } peers { permanent_uuid: "99f0ce1db669473cb520703bff465d2f" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 33047 } } peers { permanent_uuid: "83f908250a6b4cdc9f27b1141221d4d9" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 34927 } }
I20260806 03:47:20.234416 23787 leader_election.cc:288] T f898c8c90ece4ce6bf460e8ef402d407 P 99f0ce1db669473cb520703bff465d2f [CANDIDATE]: Term 1 election: Requested vote from peers 997d20698f9b493eb729667604016e89 (127.11.253.195:37749), 83f908250a6b4cdc9f27b1141221d4d9 (127.11.253.194:34927)
I20260806 03:47:20.234575 23472 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f898c8c90ece4ce6bf460e8ef402d407" candidate_uuid: "99f0ce1db669473cb520703bff465d2f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f908250a6b4cdc9f27b1141221d4d9"
I20260806 03:47:20.234632 23675 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f898c8c90ece4ce6bf460e8ef402d407" candidate_uuid: "99f0ce1db669473cb520703bff465d2f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "997d20698f9b493eb729667604016e89"
I20260806 03:47:20.234679 23472 raft_consensus.cc:3037] T f898c8c90ece4ce6bf460e8ef402d407 P 83f908250a6b4cdc9f27b1141221d4d9 [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:47:20.234702 23675 raft_consensus.cc:3037] T f898c8c90ece4ce6bf460e8ef402d407 P 997d20698f9b493eb729667604016e89 [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:47:20.235590 23675 raft_consensus.cc:2454] T f898c8c90ece4ce6bf460e8ef402d407 P 997d20698f9b493eb729667604016e89 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99f0ce1db669473cb520703bff465d2f in term 1.
I20260806 03:47:20.235590 23472 raft_consensus.cc:2454] T f898c8c90ece4ce6bf460e8ef402d407 P 83f908250a6b4cdc9f27b1141221d4d9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99f0ce1db669473cb520703bff465d2f in term 1.
I20260806 03:47:20.235749 23177 leader_election.cc:302] T f898c8c90ece4ce6bf460e8ef402d407 P 99f0ce1db669473cb520703bff465d2f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f908250a6b4cdc9f27b1141221d4d9, 99f0ce1db669473cb520703bff465d2f; no voters:
I20260806 03:47:20.235885 23787 raft_consensus.cc:2781] T f898c8c90ece4ce6bf460e8ef402d407 P 99f0ce1db669473cb520703bff465d2f [term 1 FOLLOWER]: Leader election won for term 1
I20260806 03:47:20.236043 23787 raft_consensus.cc:686] T f898c8c90ece4ce6bf460e8ef402d407 P 99f0ce1db669473cb520703bff465d2f [term 1 LEADER]: Becoming Leader. State: Replica: 99f0ce1db669473cb520703bff465d2f, State: Running, Role: LEADER
I20260806 03:47:20.236166 23787 consensus_queue.cc:237] T f898c8c90ece4ce6bf460e8ef402d407 P 99f0ce1db669473cb520703bff465d2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997d20698f9b493eb729667604016e89" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 37749 } } peers { permanent_uuid: "99f0ce1db669473cb520703bff465d2f" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 33047 } } peers { permanent_uuid: "83f908250a6b4cdc9f27b1141221d4d9" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 34927 } }
I20260806 03:47:20.237087 23045 catalog_manager.cc:5495] T f898c8c90ece4ce6bf460e8ef402d407 P 99f0ce1db669473cb520703bff465d2f reported cstate change: term changed from 0 to 1, leader changed from <none> to 99f0ce1db669473cb520703bff465d2f (127.11.253.193). New cstate: current_term: 1 leader_uuid: "99f0ce1db669473cb520703bff465d2f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997d20698f9b493eb729667604016e89" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 37749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "99f0ce1db669473cb520703bff465d2f" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 33047 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "83f908250a6b4cdc9f27b1141221d4d9" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 34927 } health_report { overall_health: UNKNOWN } } }
I20260806 03:47:20.261974 23772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:20.259508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40474 (local address 127.11.253.193:33047)
0806 03:47:20.259574 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0806 03:47:20.259576 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:47:20.259585 (+ 9us) server_negotiation.cc:407] Connection header received
0806 03:47:20.259637 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:47:20.259638 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:47:20.259658 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:47:20.259742 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 03:47:20.259878 (+ 136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:20.260353 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:20.261129 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:20.261327 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:20.261403 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:20.261596 (+ 193us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 03:47:20.261702 (+ 106us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 03:47:20.261802 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0806 03:47:20.261821 (+ 19us) server_negotiation.cc:299] Negotiation successful
0806 03:47:20.261844 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260806 03:47:20.264045 23472 raft_consensus.cc:1261] T f898c8c90ece4ce6bf460e8ef402d407 P 83f908250a6b4cdc9f27b1141221d4d9 [term 1 FOLLOWER]: Refusing update from remote peer 99f0ce1db669473cb520703bff465d2f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 03:47:20.264075 23675 raft_consensus.cc:1261] T f898c8c90ece4ce6bf460e8ef402d407 P 997d20698f9b493eb729667604016e89 [term 1 FOLLOWER]: Refusing update from remote peer 99f0ce1db669473cb520703bff465d2f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 03:47:20.264448 23795 consensus_queue.cc:1035] T f898c8c90ece4ce6bf460e8ef402d407 P 99f0ce1db669473cb520703bff465d2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f908250a6b4cdc9f27b1141221d4d9" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 34927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 03:47:20.264599 23787 consensus_queue.cc:1035] T f898c8c90ece4ce6bf460e8ef402d407 P 99f0ce1db669473cb520703bff465d2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "997d20698f9b493eb729667604016e89" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 37749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 03:47:20.269102 23045 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56072:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 03:47:20.269233 23045 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56072:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 03:47:20.270051 23045 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1faf2e28f8564261b4b003157ec7c205: Sending DeleteTablet for 3 replicas of tablet f898c8c90ece4ce6bf460e8ef402d407
I20260806 03:47:20.270376 23241 tablet_service.cc:1514] Processing DeleteTablet for tablet f898c8c90ece4ce6bf460e8ef402d407 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 03:47:20 UTC) from {username='oryx', principal='oryx/127.11.253.254@KRBTEST.COM'} at 127.0.0.1:40468
I20260806 03:47:20.270380 23642 tablet_service.cc:1514] Processing DeleteTablet for tablet f898c8c90ece4ce6bf460e8ef402d407 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 03:47:20 UTC) from {username='oryx', principal='oryx/127.11.253.254@KRBTEST.COM'} at 127.0.0.1:43088
I20260806 03:47:20.270459 23438 tablet_service.cc:1514] Processing DeleteTablet for tablet f898c8c90ece4ce6bf460e8ef402d407 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 03:47:20 UTC) from {username='oryx', principal='oryx/127.11.253.254@KRBTEST.COM'} at 127.0.0.1:57708
I20260806 03:47:20.270771 23641 tablet_service.cc:1514] Processing DeleteTablet for tablet f898c8c90ece4ce6bf460e8ef402d407 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 03:47:20 UTC) from {username='oryx', principal='oryx/127.11.253.254@KRBTEST.COM'} at 127.0.0.1:43088
I20260806 03:47:20.270933 23045 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56072:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260806 03:47:20.271054 23813 tablet_replica.cc:330] T f898c8c90ece4ce6bf460e8ef402d407 P 997d20698f9b493eb729667604016e89: stopping tablet replica
W20260806 03:47:20.271085 23045 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 03:47:20.271140 23813 raft_consensus.cc:2229] T f898c8c90ece4ce6bf460e8ef402d407 P 997d20698f9b493eb729667604016e89 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 03:47:20.271302 23813 raft_consensus.cc:2258] T f898c8c90ece4ce6bf460e8ef402d407 P 997d20698f9b493eb729667604016e89 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 03:47:20.271471 23814 tablet_replica.cc:330] T f898c8c90ece4ce6bf460e8ef402d407 P 99f0ce1db669473cb520703bff465d2f: stopping tablet replica
I20260806 03:47:20.271848 23815 tablet_replica.cc:330] T f898c8c90ece4ce6bf460e8ef402d407 P 83f908250a6b4cdc9f27b1141221d4d9: stopping tablet replica
I20260806 03:47:20.272002 23814 raft_consensus.cc:2229] T f898c8c90ece4ce6bf460e8ef402d407 P 99f0ce1db669473cb520703bff465d2f [term 1 LEADER]: Raft consensus shutting down.
I20260806 03:47:20.272145 23814 raft_consensus.cc:2258] T f898c8c90ece4ce6bf460e8ef402d407 P 99f0ce1db669473cb520703bff465d2f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 03:47:20.272297 23641 tablet_service.cc:1467] Processing CreateTablet for tablet a1a1c59d5ae242878165796d2c58be06 (DEFAULT_TABLE table=test-table [id=fe6e4106ef614ee296745c9240bc65f1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 03:47:20.272322 23815 raft_consensus.cc:2229] T f898c8c90ece4ce6bf460e8ef402d407 P 83f908250a6b4cdc9f27b1141221d4d9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 03:47:20.272423 23641 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a1a1c59d5ae242878165796d2c58be06. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:47:20.272433 23815 raft_consensus.cc:2258] T f898c8c90ece4ce6bf460e8ef402d407 P 83f908250a6b4cdc9f27b1141221d4d9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 03:47:20.272406 23241 tablet_service.cc:1467] Processing CreateTablet for tablet a1a1c59d5ae242878165796d2c58be06 (DEFAULT_TABLE table=test-table [id=fe6e4106ef614ee296745c9240bc65f1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 03:47:20.272605 23241 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a1a1c59d5ae242878165796d2c58be06. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:47:20.272912 23815 ts_tablet_manager.cc:1905] T f898c8c90ece4ce6bf460e8ef402d407 P 83f908250a6b4cdc9f27b1141221d4d9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 03:47:20.273976 23776 tablet_bootstrap.cc:492] T a1a1c59d5ae242878165796d2c58be06 P 99f0ce1db669473cb520703bff465d2f: Bootstrap starting.
I20260806 03:47:20.274055 23438 tablet_service.cc:1467] Processing CreateTablet for tablet a1a1c59d5ae242878165796d2c58be06 (DEFAULT_TABLE table=test-table [id=fe6e4106ef614ee296745c9240bc65f1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 03:47:20.274156 23438 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a1a1c59d5ae242878165796d2c58be06. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:47:20.274432 23776 tablet_bootstrap.cc:654] T a1a1c59d5ae242878165796d2c58be06 P 99f0ce1db669473cb520703bff465d2f: Neither blocks nor log segments found. Creating new log.
I20260806 03:47:20.274899 23815 ts_tablet_manager.cc:1918] T f898c8c90ece4ce6bf460e8ef402d407 P 83f908250a6b4cdc9f27b1141221d4d9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 03:47:20.275004 23815 log.cc:1196] T f898c8c90ece4ce6bf460e8ef402d407 P 83f908250a6b4cdc9f27b1141221d4d9: Deleting WAL directory at /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785988004637995-12279-0/minicluster-data/ts-1/wal/wals/f898c8c90ece4ce6bf460e8ef402d407
I20260806 03:47:20.275298 23815 ts_tablet_manager.cc:1939] T f898c8c90ece4ce6bf460e8ef402d407 P 83f908250a6b4cdc9f27b1141221d4d9: Deleting consensus metadata
I20260806 03:47:20.275449 23813 ts_tablet_manager.cc:1905] T f898c8c90ece4ce6bf460e8ef402d407 P 997d20698f9b493eb729667604016e89: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 03:47:20.275484 23814 ts_tablet_manager.cc:1905] T f898c8c90ece4ce6bf460e8ef402d407 P 99f0ce1db669473cb520703bff465d2f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 03:47:20.275530 23779 tablet_bootstrap.cc:492] T a1a1c59d5ae242878165796d2c58be06 P 83f908250a6b4cdc9f27b1141221d4d9: Bootstrap starting.
I20260806 03:47:20.275929 23779 tablet_bootstrap.cc:654] T a1a1c59d5ae242878165796d2c58be06 P 83f908250a6b4cdc9f27b1141221d4d9: Neither blocks nor log segments found. Creating new log.
I20260806 03:47:20.276352 23029 catalog_manager.cc:4841] TS 83f908250a6b4cdc9f27b1141221d4d9 (127.11.253.194:34927): tablet f898c8c90ece4ce6bf460e8ef402d407 (table test-table [id=4c200faa76214c5eaa1122547c47d95c]) successfully deleted
I20260806 03:47:20.276719 23779 tablet_bootstrap.cc:492] T a1a1c59d5ae242878165796d2c58be06 P 83f908250a6b4cdc9f27b1141221d4d9: No bootstrap required, opened a new log
I20260806 03:47:20.276772 23779 ts_tablet_manager.cc:1397] T a1a1c59d5ae242878165796d2c58be06 P 83f908250a6b4cdc9f27b1141221d4d9: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260806 03:47:20.276934 23779 raft_consensus.cc:348] T a1a1c59d5ae242878165796d2c58be06 P 83f908250a6b4cdc9f27b1141221d4d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99f0ce1db669473cb520703bff465d2f" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 33047 } } peers { permanent_uuid: "83f908250a6b4cdc9f27b1141221d4d9" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 34927 } } peers { permanent_uuid: "997d20698f9b493eb729667604016e89" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 37749 } }
I20260806 03:47:20.277006 23779 raft_consensus.cc:374] T a1a1c59d5ae242878165796d2c58be06 P 83f908250a6b4cdc9f27b1141221d4d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:47:20.277021 23779 raft_consensus.cc:729] T a1a1c59d5ae242878165796d2c58be06 P 83f908250a6b4cdc9f27b1141221d4d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83f908250a6b4cdc9f27b1141221d4d9, State: Initialized, Role: FOLLOWER
I20260806 03:47:20.277056 23779 consensus_queue.cc:260] T a1a1c59d5ae242878165796d2c58be06 P 83f908250a6b4cdc9f27b1141221d4d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99f0ce1db669473cb520703bff465d2f" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 33047 } } peers { permanent_uuid: "83f908250a6b4cdc9f27b1141221d4d9" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 34927 } } peers { permanent_uuid: "997d20698f9b493eb729667604016e89" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 37749 } }
I20260806 03:47:20.277089 23813 ts_tablet_manager.cc:1918] T f898c8c90ece4ce6bf460e8ef402d407 P 997d20698f9b493eb729667604016e89: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 03:47:20.277154 23813 log.cc:1196] T f898c8c90ece4ce6bf460e8ef402d407 P 997d20698f9b493eb729667604016e89: Deleting WAL directory at /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785988004637995-12279-0/minicluster-data/ts-2/wal/wals/f898c8c90ece4ce6bf460e8ef402d407
I20260806 03:47:20.277148 23779 ts_tablet_manager.cc:1428] T a1a1c59d5ae242878165796d2c58be06 P 83f908250a6b4cdc9f27b1141221d4d9: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 03:47:20.277355 23777 tablet_bootstrap.cc:492] T a1a1c59d5ae242878165796d2c58be06 P 997d20698f9b493eb729667604016e89: Bootstrap starting.
I20260806 03:47:20.277420 23813 ts_tablet_manager.cc:1939] T f898c8c90ece4ce6bf460e8ef402d407 P 997d20698f9b493eb729667604016e89: Deleting consensus metadata
I20260806 03:47:20.277765 23777 tablet_bootstrap.cc:654] T a1a1c59d5ae242878165796d2c58be06 P 997d20698f9b493eb729667604016e89: Neither blocks nor log segments found. Creating new log.
I20260806 03:47:20.277797 23031 catalog_manager.cc:4841] TS 997d20698f9b493eb729667604016e89 (127.11.253.195:37749): tablet f898c8c90ece4ce6bf460e8ef402d407 (table test-table [id=4c200faa76214c5eaa1122547c47d95c]) successfully deleted
I20260806 03:47:20.278139 23776 tablet_bootstrap.cc:492] T a1a1c59d5ae242878165796d2c58be06 P 99f0ce1db669473cb520703bff465d2f: No bootstrap required, opened a new log
W20260806 03:47:20.278162 23031 catalog_manager.cc:4808] TS 997d20698f9b493eb729667604016e89 (127.11.253.195:37749): delete failed for tablet f898c8c90ece4ce6bf460e8ef402d407 because the tablet was not found. No further retry: Not found: Tablet not found: f898c8c90ece4ce6bf460e8ef402d407
I20260806 03:47:20.278195 23776 ts_tablet_manager.cc:1397] T a1a1c59d5ae242878165796d2c58be06 P 99f0ce1db669473cb520703bff465d2f: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260806 03:47:20.278363 23777 tablet_bootstrap.cc:492] T a1a1c59d5ae242878165796d2c58be06 P 997d20698f9b493eb729667604016e89: No bootstrap required, opened a new log
I20260806 03:47:20.278407 23777 ts_tablet_manager.cc:1397] T a1a1c59d5ae242878165796d2c58be06 P 997d20698f9b493eb729667604016e89: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260806 03:47:20.278386 23776 raft_consensus.cc:348] T a1a1c59d5ae242878165796d2c58be06 P 99f0ce1db669473cb520703bff465d2f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99f0ce1db669473cb520703bff465d2f" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 33047 } } peers { permanent_uuid: "83f908250a6b4cdc9f27b1141221d4d9" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 34927 } } peers { permanent_uuid: "997d20698f9b493eb729667604016e89" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 37749 } }
I20260806 03:47:20.278447 23776 raft_consensus.cc:374] T a1a1c59d5ae242878165796d2c58be06 P 99f0ce1db669473cb520703bff465d2f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:47:20.278468 23776 raft_consensus.cc:729] T a1a1c59d5ae242878165796d2c58be06 P 99f0ce1db669473cb520703bff465d2f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99f0ce1db669473cb520703bff465d2f, State: Initialized, Role: FOLLOWER
I20260806 03:47:20.278517 23776 consensus_queue.cc:260] T a1a1c59d5ae242878165796d2c58be06 P 99f0ce1db669473cb520703bff465d2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99f0ce1db669473cb520703bff465d2f" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 33047 } } peers { permanent_uuid: "83f908250a6b4cdc9f27b1141221d4d9" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 34927 } } peers { permanent_uuid: "997d20698f9b493eb729667604016e89" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 37749 } }
I20260806 03:47:20.278625 23776 ts_tablet_manager.cc:1428] T a1a1c59d5ae242878165796d2c58be06 P 99f0ce1db669473cb520703bff465d2f: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 03:47:20.278601 23777 raft_consensus.cc:348] T a1a1c59d5ae242878165796d2c58be06 P 997d20698f9b493eb729667604016e89 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99f0ce1db669473cb520703bff465d2f" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 33047 } } peers { permanent_uuid: "83f908250a6b4cdc9f27b1141221d4d9" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 34927 } } peers { permanent_uuid: "997d20698f9b493eb729667604016e89" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 37749 } }
I20260806 03:47:20.278669 23777 raft_consensus.cc:374] T a1a1c59d5ae242878165796d2c58be06 P 997d20698f9b493eb729667604016e89 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:47:20.278692 23777 raft_consensus.cc:729] T a1a1c59d5ae242878165796d2c58be06 P 997d20698f9b493eb729667604016e89 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 997d20698f9b493eb729667604016e89, State: Initialized, Role: FOLLOWER
I20260806 03:47:20.278746 23777 consensus_queue.cc:260] T a1a1c59d5ae242878165796d2c58be06 P 997d20698f9b493eb729667604016e89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99f0ce1db669473cb520703bff465d2f" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 33047 } } peers { permanent_uuid: "83f908250a6b4cdc9f27b1141221d4d9" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 34927 } } peers { permanent_uuid: "997d20698f9b493eb729667604016e89" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 37749 } }
I20260806 03:47:20.278843 23777 ts_tablet_manager.cc:1428] T a1a1c59d5ae242878165796d2c58be06 P 997d20698f9b493eb729667604016e89: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260806 03:47:20.279686 23814 ts_tablet_manager.cc:1918] T f898c8c90ece4ce6bf460e8ef402d407 P 99f0ce1db669473cb520703bff465d2f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 03:47:20.279807 23814 log.cc:1196] T f898c8c90ece4ce6bf460e8ef402d407 P 99f0ce1db669473cb520703bff465d2f: Deleting WAL directory at /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785988004637995-12279-0/minicluster-data/ts-0/wal/wals/f898c8c90ece4ce6bf460e8ef402d407
I20260806 03:47:20.280037 23814 ts_tablet_manager.cc:1939] T f898c8c90ece4ce6bf460e8ef402d407 P 99f0ce1db669473cb520703bff465d2f: Deleting consensus metadata
I20260806 03:47:20.280344 23029 catalog_manager.cc:4841] TS 99f0ce1db669473cb520703bff465d2f (127.11.253.193:33047): tablet f898c8c90ece4ce6bf460e8ef402d407 (table test-table [id=4c200faa76214c5eaa1122547c47d95c]) successfully deleted
I20260806 03:47:20.318306 23810 raft_consensus.cc:482] T a1a1c59d5ae242878165796d2c58be06 P 99f0ce1db669473cb520703bff465d2f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 03:47:20.318401 23810 raft_consensus.cc:504] T a1a1c59d5ae242878165796d2c58be06 P 99f0ce1db669473cb520703bff465d2f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99f0ce1db669473cb520703bff465d2f" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 33047 } } peers { permanent_uuid: "83f908250a6b4cdc9f27b1141221d4d9" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 34927 } } peers { permanent_uuid: "997d20698f9b493eb729667604016e89" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 37749 } }
I20260806 03:47:20.318616 23810 leader_election.cc:288] T a1a1c59d5ae242878165796d2c58be06 P 99f0ce1db669473cb520703bff465d2f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 83f908250a6b4cdc9f27b1141221d4d9 (127.11.253.194:34927), 997d20698f9b493eb729667604016e89 (127.11.253.195:37749)
I20260806 03:47:20.318837 23675 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a1a1c59d5ae242878165796d2c58be06" candidate_uuid: "99f0ce1db669473cb520703bff465d2f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "997d20698f9b493eb729667604016e89" is_pre_election: true
I20260806 03:47:20.318837 23472 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a1a1c59d5ae242878165796d2c58be06" candidate_uuid: "99f0ce1db669473cb520703bff465d2f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f908250a6b4cdc9f27b1141221d4d9" is_pre_election: true
I20260806 03:47:20.318944 23675 raft_consensus.cc:2454] T a1a1c59d5ae242878165796d2c58be06 P 997d20698f9b493eb729667604016e89 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99f0ce1db669473cb520703bff465d2f in term 0.
I20260806 03:47:20.318944 23472 raft_consensus.cc:2454] T a1a1c59d5ae242878165796d2c58be06 P 83f908250a6b4cdc9f27b1141221d4d9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99f0ce1db669473cb520703bff465d2f in term 0.
I20260806 03:47:20.319416 23178 leader_election.cc:302] T a1a1c59d5ae242878165796d2c58be06 P 99f0ce1db669473cb520703bff465d2f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 997d20698f9b493eb729667604016e89, 99f0ce1db669473cb520703bff465d2f; no voters:
I20260806 03:47:20.319552 23810 raft_consensus.cc:2781] T a1a1c59d5ae242878165796d2c58be06 P 99f0ce1db669473cb520703bff465d2f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 03:47:20.319599 23810 raft_consensus.cc:482] T a1a1c59d5ae242878165796d2c58be06 P 99f0ce1db669473cb520703bff465d2f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 03:47:20.319614 23810 raft_consensus.cc:3037] T a1a1c59d5ae242878165796d2c58be06 P 99f0ce1db669473cb520703bff465d2f [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:47:20.320385 23810 raft_consensus.cc:504] T a1a1c59d5ae242878165796d2c58be06 P 99f0ce1db669473cb520703bff465d2f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99f0ce1db669473cb520703bff465d2f" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 33047 } } peers { permanent_uuid: "83f908250a6b4cdc9f27b1141221d4d9" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 34927 } } peers { permanent_uuid: "997d20698f9b493eb729667604016e89" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 37749 } }
I20260806 03:47:20.320528 23810 leader_election.cc:288] T a1a1c59d5ae242878165796d2c58be06 P 99f0ce1db669473cb520703bff465d2f [CANDIDATE]: Term 1 election: Requested vote from peers 83f908250a6b4cdc9f27b1141221d4d9 (127.11.253.194:34927), 997d20698f9b493eb729667604016e89 (127.11.253.195:37749)
I20260806 03:47:20.320657 23472 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a1a1c59d5ae242878165796d2c58be06" candidate_uuid: "99f0ce1db669473cb520703bff465d2f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f908250a6b4cdc9f27b1141221d4d9"
I20260806 03:47:20.320741 23472 raft_consensus.cc:3037] T a1a1c59d5ae242878165796d2c58be06 P 83f908250a6b4cdc9f27b1141221d4d9 [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:47:20.320752 23675 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a1a1c59d5ae242878165796d2c58be06" candidate_uuid: "99f0ce1db669473cb520703bff465d2f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "997d20698f9b493eb729667604016e89"
I20260806 03:47:20.320816 23675 raft_consensus.cc:3037] T a1a1c59d5ae242878165796d2c58be06 P 997d20698f9b493eb729667604016e89 [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:47:20.321521 23472 raft_consensus.cc:2454] T a1a1c59d5ae242878165796d2c58be06 P 83f908250a6b4cdc9f27b1141221d4d9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99f0ce1db669473cb520703bff465d2f in term 1.
I20260806 03:47:20.321575 23675 raft_consensus.cc:2454] T a1a1c59d5ae242878165796d2c58be06 P 997d20698f9b493eb729667604016e89 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99f0ce1db669473cb520703bff465d2f in term 1.
I20260806 03:47:20.321722 23177 leader_election.cc:302] T a1a1c59d5ae242878165796d2c58be06 P 99f0ce1db669473cb520703bff465d2f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f908250a6b4cdc9f27b1141221d4d9, 99f0ce1db669473cb520703bff465d2f; no voters:
I20260806 03:47:20.321820 23810 raft_consensus.cc:2781] T a1a1c59d5ae242878165796d2c58be06 P 99f0ce1db669473cb520703bff465d2f [term 1 FOLLOWER]: Leader election won for term 1
I20260806 03:47:20.321908 23810 raft_consensus.cc:686] T a1a1c59d5ae242878165796d2c58be06 P 99f0ce1db669473cb520703bff465d2f [term 1 LEADER]: Becoming Leader. State: Replica: 99f0ce1db669473cb520703bff465d2f, State: Running, Role: LEADER
I20260806 03:47:20.321982 23810 consensus_queue.cc:237] T a1a1c59d5ae242878165796d2c58be06 P 99f0ce1db669473cb520703bff465d2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99f0ce1db669473cb520703bff465d2f" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 33047 } } peers { permanent_uuid: "83f908250a6b4cdc9f27b1141221d4d9" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 34927 } } peers { permanent_uuid: "997d20698f9b493eb729667604016e89" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 37749 } }
I20260806 03:47:20.322543 23048 catalog_manager.cc:5495] T a1a1c59d5ae242878165796d2c58be06 P 99f0ce1db669473cb520703bff465d2f reported cstate change: term changed from 0 to 1, leader changed from <none> to 99f0ce1db669473cb520703bff465d2f (127.11.253.193). New cstate: current_term: 1 leader_uuid: "99f0ce1db669473cb520703bff465d2f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99f0ce1db669473cb520703bff465d2f" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 33047 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "83f908250a6b4cdc9f27b1141221d4d9" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 34927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "997d20698f9b493eb729667604016e89" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 37749 } health_report { overall_health: UNKNOWN } } }
Aug 06 03:47:20 dist-test-slave-6ntk krb5kdc[22990](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988039, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.253.254@KRBTEST.COM for oryx/127.11.253.254@KRBTEST.COM
I20260806 03:47:20.363197 23830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:20.354870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:38857 (local address 127.0.0.1:56084)
0806 03:47:20.355068 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0806 03:47:20.355081 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 03:47:20.355170 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:47:20.355401 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:47:20.355402 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:47:20.355845 (+ 443us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:47:20.355907 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:20.355910 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:20.356657 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:20.356658 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:20.357532 (+ 874us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:20.357536 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:20.357960 (+ 424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:20.357961 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:20.358026 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:20.358629 (+ 603us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:47:20.358635 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:47:20.360253 (+ 1618us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:47:20.362023 (+ 1770us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:47:20.362032 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:47:20.362034 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:47:20.362248 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:47:20.362456 (+ 208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:47:20.362456 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:47:20.362457 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:47:20.362523 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:47:20.362779 (+ 256us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:47:20.362788 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:47:20.362834 (+ 46us) client_negotiation.cc:769] Sending connection context
0806 03:47:20.362911 (+ 77us) client_negotiation.cc:240] Negotiation successful
0806 03:47:20.363067 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":99,"threads_started":1}
I20260806 03:47:20.363215 23127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:20.354957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56084 (local address 127.11.253.254:38857)
0806 03:47:20.355043 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0806 03:47:20.355046 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 03:47:20.355164 (+ 118us) server_negotiation.cc:407] Connection header received
0806 03:47:20.355282 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:47:20.355283 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:47:20.355295 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:47:20.355360 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:47:20.356017 (+ 657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:20.356560 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:20.357625 (+ 1065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:20.357808 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:20.357863 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:20.360362 (+ 2499us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:47:20.360364 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:47:20.360364 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:47:20.360372 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:47:20.361931 (+ 1559us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:47:20.362326 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:47:20.362327 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:47:20.362328 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:47:20.362389 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:47:20.362591 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:47:20.362592 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:47:20.362592 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:47:20.362705 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:47:20.362776 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0806 03:47:20.362948 (+ 172us) server_negotiation.cc:299] Negotiation successful
0806 03:47:20.363122 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":103,"server-negotiator.queue_time_us":53}
I20260806 03:47:20.365425 23048 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.11.253.254@KRBTEST.COM'} at 127.0.0.1:56084:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260806 03:47:20.365607 23048 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 03:47:20.367316 23241 tablet_service.cc:1467] Processing CreateTablet for tablet 379ee64748b24477918b5608950d6d31 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=cbb973be8a6b4015b9a359ddab7749be]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 03:47:20.367316 23438 tablet_service.cc:1467] Processing CreateTablet for tablet 379ee64748b24477918b5608950d6d31 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=cbb973be8a6b4015b9a359ddab7749be]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 03:47:20.367483 23241 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 379ee64748b24477918b5608950d6d31. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:47:20.367484 23438 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 379ee64748b24477918b5608950d6d31. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:47:20.369007 23776 tablet_bootstrap.cc:492] T 379ee64748b24477918b5608950d6d31 P 99f0ce1db669473cb520703bff465d2f: Bootstrap starting.
I20260806 03:47:20.369131 23641 tablet_service.cc:1467] Processing CreateTablet for tablet 379ee64748b24477918b5608950d6d31 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=cbb973be8a6b4015b9a359ddab7749be]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 03:47:20.369274 23641 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 379ee64748b24477918b5608950d6d31. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:47:20.369791 23776 tablet_bootstrap.cc:654] T 379ee64748b24477918b5608950d6d31 P 99f0ce1db669473cb520703bff465d2f: Neither blocks nor log segments found. Creating new log.
I20260806 03:47:20.370182 23779 tablet_bootstrap.cc:492] T 379ee64748b24477918b5608950d6d31 P 83f908250a6b4cdc9f27b1141221d4d9: Bootstrap starting.
I20260806 03:47:20.370482 23776 tablet_bootstrap.cc:492] T 379ee64748b24477918b5608950d6d31 P 99f0ce1db669473cb520703bff465d2f: No bootstrap required, opened a new log
I20260806 03:47:20.370530 23776 ts_tablet_manager.cc:1397] T 379ee64748b24477918b5608950d6d31 P 99f0ce1db669473cb520703bff465d2f: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260806 03:47:20.370572 23777 tablet_bootstrap.cc:492] T 379ee64748b24477918b5608950d6d31 P 997d20698f9b493eb729667604016e89: Bootstrap starting.
I20260806 03:47:20.370625 23779 tablet_bootstrap.cc:654] T 379ee64748b24477918b5608950d6d31 P 83f908250a6b4cdc9f27b1141221d4d9: Neither blocks nor log segments found. Creating new log.
I20260806 03:47:20.370687 23776 raft_consensus.cc:348] T 379ee64748b24477918b5608950d6d31 P 99f0ce1db669473cb520703bff465d2f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99f0ce1db669473cb520703bff465d2f" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 33047 } } peers { permanent_uuid: "83f908250a6b4cdc9f27b1141221d4d9" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 34927 } } peers { permanent_uuid: "997d20698f9b493eb729667604016e89" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 37749 } }
I20260806 03:47:20.370787 23776 raft_consensus.cc:374] T 379ee64748b24477918b5608950d6d31 P 99f0ce1db669473cb520703bff465d2f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:47:20.370808 23776 raft_consensus.cc:729] T 379ee64748b24477918b5608950d6d31 P 99f0ce1db669473cb520703bff465d2f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99f0ce1db669473cb520703bff465d2f, State: Initialized, Role: FOLLOWER
I20260806 03:47:20.370857 23776 consensus_queue.cc:260] T 379ee64748b24477918b5608950d6d31 P 99f0ce1db669473cb520703bff465d2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99f0ce1db669473cb520703bff465d2f" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 33047 } } peers { permanent_uuid: "83f908250a6b4cdc9f27b1141221d4d9" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 34927 } } peers { permanent_uuid: "997d20698f9b493eb729667604016e89" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 37749 } }
I20260806 03:47:20.370957 23776 ts_tablet_manager.cc:1428] T 379ee64748b24477918b5608950d6d31 P 99f0ce1db669473cb520703bff465d2f: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260806 03:47:20.370997 23777 tablet_bootstrap.cc:654] T 379ee64748b24477918b5608950d6d31 P 997d20698f9b493eb729667604016e89: Neither blocks nor log segments found. Creating new log.
I20260806 03:47:20.371374 23787 tablet_replica.cc:439] T 379ee64748b24477918b5608950d6d31 P 99f0ce1db669473cb520703bff465d2f: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99f0ce1db669473cb520703bff465d2f" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 33047 } } peers { permanent_uuid: "83f908250a6b4cdc9f27b1141221d4d9" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 34927 } } peers { permanent_uuid: "997d20698f9b493eb729667604016e89" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 37749 } } }
I20260806 03:47:20.371471 23787 tablet_replica.cc:442] T 379ee64748b24477918b5608950d6d31 P 99f0ce1db669473cb520703bff465d2f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 03:47:20.371918 23777 tablet_bootstrap.cc:492] T 379ee64748b24477918b5608950d6d31 P 997d20698f9b493eb729667604016e89: No bootstrap required, opened a new log
I20260806 03:47:20.371970 23777 ts_tablet_manager.cc:1397] T 379ee64748b24477918b5608950d6d31 P 997d20698f9b493eb729667604016e89: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260806 03:47:20.372121 23777 raft_consensus.cc:348] T 379ee64748b24477918b5608950d6d31 P 997d20698f9b493eb729667604016e89 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99f0ce1db669473cb520703bff465d2f" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 33047 } } peers { permanent_uuid: "83f908250a6b4cdc9f27b1141221d4d9" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 34927 } } peers { permanent_uuid: "997d20698f9b493eb729667604016e89" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 37749 } }
I20260806 03:47:20.372208 23777 raft_consensus.cc:374] T 379ee64748b24477918b5608950d6d31 P 997d20698f9b493eb729667604016e89 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:47:20.372231 23777 raft_consensus.cc:729] T 379ee64748b24477918b5608950d6d31 P 997d20698f9b493eb729667604016e89 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 997d20698f9b493eb729667604016e89, State: Initialized, Role: FOLLOWER
I20260806 03:47:20.372242 23779 tablet_bootstrap.cc:492] T 379ee64748b24477918b5608950d6d31 P 83f908250a6b4cdc9f27b1141221d4d9: No bootstrap required, opened a new log
I20260806 03:47:20.372287 23779 ts_tablet_manager.cc:1397] T 379ee64748b24477918b5608950d6d31 P 83f908250a6b4cdc9f27b1141221d4d9: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 03:47:20.372282 23777 consensus_queue.cc:260] T 379ee64748b24477918b5608950d6d31 P 997d20698f9b493eb729667604016e89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99f0ce1db669473cb520703bff465d2f" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 33047 } } peers { permanent_uuid: "83f908250a6b4cdc9f27b1141221d4d9" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 34927 } } peers { permanent_uuid: "997d20698f9b493eb729667604016e89" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 37749 } }
I20260806 03:47:20.372382 23777 ts_tablet_manager.cc:1428] T 379ee64748b24477918b5608950d6d31 P 997d20698f9b493eb729667604016e89: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260806 03:47:20.372437 23779 raft_consensus.cc:348] T 379ee64748b24477918b5608950d6d31 P 83f908250a6b4cdc9f27b1141221d4d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99f0ce1db669473cb520703bff465d2f" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 33047 } } peers { permanent_uuid: "83f908250a6b4cdc9f27b1141221d4d9" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 34927 } } peers { permanent_uuid: "997d20698f9b493eb729667604016e89" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 37749 } }
I20260806 03:47:20.372514 23779 raft_consensus.cc:374] T 379ee64748b24477918b5608950d6d31 P 83f908250a6b4cdc9f27b1141221d4d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:47:20.372536 23779 raft_consensus.cc:729] T 379ee64748b24477918b5608950d6d31 P 83f908250a6b4cdc9f27b1141221d4d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83f908250a6b4cdc9f27b1141221d4d9, State: Initialized, Role: FOLLOWER
I20260806 03:47:20.372509 23786 tablet_replica.cc:439] T 379ee64748b24477918b5608950d6d31 P 997d20698f9b493eb729667604016e89: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99f0ce1db669473cb520703bff465d2f" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 33047 } } peers { permanent_uuid: "83f908250a6b4cdc9f27b1141221d4d9" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 34927 } } peers { permanent_uuid: "997d20698f9b493eb729667604016e89" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 37749 } } }
I20260806 03:47:20.372581 23786 tablet_replica.cc:442] T 379ee64748b24477918b5608950d6d31 P 997d20698f9b493eb729667604016e89: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 03:47:20.372584 23779 consensus_queue.cc:260] T 379ee64748b24477918b5608950d6d31 P 83f908250a6b4cdc9f27b1141221d4d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99f0ce1db669473cb520703bff465d2f" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 33047 } } peers { permanent_uuid: "83f908250a6b4cdc9f27b1141221d4d9" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 34927 } } peers { permanent_uuid: "997d20698f9b493eb729667604016e89" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 37749 } }
I20260806 03:47:20.372684 23779 ts_tablet_manager.cc:1428] T 379ee64748b24477918b5608950d6d31 P 83f908250a6b4cdc9f27b1141221d4d9: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 03:47:20.372949 23788 tablet_replica.cc:439] T 379ee64748b24477918b5608950d6d31 P 83f908250a6b4cdc9f27b1141221d4d9: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99f0ce1db669473cb520703bff465d2f" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 33047 } } peers { permanent_uuid: "83f908250a6b4cdc9f27b1141221d4d9" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 34927 } } peers { permanent_uuid: "997d20698f9b493eb729667604016e89" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 37749 } } }
I20260806 03:47:20.373051 23788 tablet_replica.cc:442] T 379ee64748b24477918b5608950d6d31 P 83f908250a6b4cdc9f27b1141221d4d9: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 03:47:20.706269 23787 raft_consensus.cc:482] T 379ee64748b24477918b5608950d6d31 P 99f0ce1db669473cb520703bff465d2f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 03:47:20.706396 23787 raft_consensus.cc:504] T 379ee64748b24477918b5608950d6d31 P 99f0ce1db669473cb520703bff465d2f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99f0ce1db669473cb520703bff465d2f" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 33047 } } peers { permanent_uuid: "83f908250a6b4cdc9f27b1141221d4d9" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 34927 } } peers { permanent_uuid: "997d20698f9b493eb729667604016e89" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 37749 } }
I20260806 03:47:20.706585 23787 leader_election.cc:288] T 379ee64748b24477918b5608950d6d31 P 99f0ce1db669473cb520703bff465d2f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 83f908250a6b4cdc9f27b1141221d4d9 (127.11.253.194:34927), 997d20698f9b493eb729667604016e89 (127.11.253.195:37749)
I20260806 03:47:20.706893 23675 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "379ee64748b24477918b5608950d6d31" candidate_uuid: "99f0ce1db669473cb520703bff465d2f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "997d20698f9b493eb729667604016e89" is_pre_election: true
I20260806 03:47:20.706883 23472 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "379ee64748b24477918b5608950d6d31" candidate_uuid: "99f0ce1db669473cb520703bff465d2f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f908250a6b4cdc9f27b1141221d4d9" is_pre_election: true
I20260806 03:47:20.706987 23675 raft_consensus.cc:2454] T 379ee64748b24477918b5608950d6d31 P 997d20698f9b493eb729667604016e89 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99f0ce1db669473cb520703bff465d2f in term 0.
I20260806 03:47:20.707016 23472 raft_consensus.cc:2454] T 379ee64748b24477918b5608950d6d31 P 83f908250a6b4cdc9f27b1141221d4d9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99f0ce1db669473cb520703bff465d2f in term 0.
I20260806 03:47:20.707222 23178 leader_election.cc:302] T 379ee64748b24477918b5608950d6d31 P 99f0ce1db669473cb520703bff465d2f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 83f908250a6b4cdc9f27b1141221d4d9, 997d20698f9b493eb729667604016e89, 99f0ce1db669473cb520703bff465d2f; no voters:
I20260806 03:47:20.707362 23787 raft_consensus.cc:2781] T 379ee64748b24477918b5608950d6d31 P 99f0ce1db669473cb520703bff465d2f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 03:47:20.707430 23787 raft_consensus.cc:482] T 379ee64748b24477918b5608950d6d31 P 99f0ce1db669473cb520703bff465d2f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 03:47:20.707460 23787 raft_consensus.cc:3037] T 379ee64748b24477918b5608950d6d31 P 99f0ce1db669473cb520703bff465d2f [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:47:20.708191 23787 raft_consensus.cc:504] T 379ee64748b24477918b5608950d6d31 P 99f0ce1db669473cb520703bff465d2f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99f0ce1db669473cb520703bff465d2f" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 33047 } } peers { permanent_uuid: "83f908250a6b4cdc9f27b1141221d4d9" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 34927 } } peers { permanent_uuid: "997d20698f9b493eb729667604016e89" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 37749 } }
I20260806 03:47:20.708357 23787 leader_election.cc:288] T 379ee64748b24477918b5608950d6d31 P 99f0ce1db669473cb520703bff465d2f [CANDIDATE]: Term 1 election: Requested vote from peers 83f908250a6b4cdc9f27b1141221d4d9 (127.11.253.194:34927), 997d20698f9b493eb729667604016e89 (127.11.253.195:37749)
I20260806 03:47:20.708544 23472 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "379ee64748b24477918b5608950d6d31" candidate_uuid: "99f0ce1db669473cb520703bff465d2f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f908250a6b4cdc9f27b1141221d4d9"
I20260806 03:47:20.708673 23472 raft_consensus.cc:3037] T 379ee64748b24477918b5608950d6d31 P 83f908250a6b4cdc9f27b1141221d4d9 [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:47:20.708637 23675 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "379ee64748b24477918b5608950d6d31" candidate_uuid: "99f0ce1db669473cb520703bff465d2f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "997d20698f9b493eb729667604016e89"
I20260806 03:47:20.708709 23675 raft_consensus.cc:3037] T 379ee64748b24477918b5608950d6d31 P 997d20698f9b493eb729667604016e89 [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:47:20.709470 23675 raft_consensus.cc:2454] T 379ee64748b24477918b5608950d6d31 P 997d20698f9b493eb729667604016e89 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99f0ce1db669473cb520703bff465d2f in term 1.
I20260806 03:47:20.709470 23472 raft_consensus.cc:2454] T 379ee64748b24477918b5608950d6d31 P 83f908250a6b4cdc9f27b1141221d4d9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99f0ce1db669473cb520703bff465d2f in term 1.
I20260806 03:47:20.709654 23177 leader_election.cc:302] T 379ee64748b24477918b5608950d6d31 P 99f0ce1db669473cb520703bff465d2f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f908250a6b4cdc9f27b1141221d4d9, 99f0ce1db669473cb520703bff465d2f; no voters:
I20260806 03:47:20.709793 23787 raft_consensus.cc:2781] T 379ee64748b24477918b5608950d6d31 P 99f0ce1db669473cb520703bff465d2f [term 1 FOLLOWER]: Leader election won for term 1
I20260806 03:47:20.709903 23787 raft_consensus.cc:686] T 379ee64748b24477918b5608950d6d31 P 99f0ce1db669473cb520703bff465d2f [term 1 LEADER]: Becoming Leader. State: Replica: 99f0ce1db669473cb520703bff465d2f, State: Running, Role: LEADER
I20260806 03:47:20.709969 23787 consensus_queue.cc:237] T 379ee64748b24477918b5608950d6d31 P 99f0ce1db669473cb520703bff465d2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99f0ce1db669473cb520703bff465d2f" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 33047 } } peers { permanent_uuid: "83f908250a6b4cdc9f27b1141221d4d9" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 34927 } } peers { permanent_uuid: "997d20698f9b493eb729667604016e89" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 37749 } }
I20260806 03:47:20.710265 23810 tablet_replica.cc:439] T 379ee64748b24477918b5608950d6d31 P 99f0ce1db669473cb520703bff465d2f: TxnStatusTablet state changed. Reason: New leader 99f0ce1db669473cb520703bff465d2f. Latest consensus state: current_term: 1 leader_uuid: "99f0ce1db669473cb520703bff465d2f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99f0ce1db669473cb520703bff465d2f" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 33047 } } peers { permanent_uuid: "83f908250a6b4cdc9f27b1141221d4d9" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 34927 } } peers { permanent_uuid: "997d20698f9b493eb729667604016e89" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 37749 } } }
I20260806 03:47:20.710358 23810 tablet_replica.cc:442] T 379ee64748b24477918b5608950d6d31 P 99f0ce1db669473cb520703bff465d2f: This TxnStatusTablet replica's current role is: LEADER
I20260806 03:47:20.710577 23835 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260806 03:47:20.710618 23045 catalog_manager.cc:5495] T 379ee64748b24477918b5608950d6d31 P 99f0ce1db669473cb520703bff465d2f reported cstate change: term changed from 0 to 1, leader changed from <none> to 99f0ce1db669473cb520703bff465d2f (127.11.253.193). New cstate: current_term: 1 leader_uuid: "99f0ce1db669473cb520703bff465d2f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99f0ce1db669473cb520703bff465d2f" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 33047 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "83f908250a6b4cdc9f27b1141221d4d9" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 34927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "997d20698f9b493eb729667604016e89" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 37749 } health_report { overall_health: UNKNOWN } } }
I20260806 03:47:20.710739 23835 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260806 03:47:20.725503 23830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:20.723074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.193:33047 (local address 127.0.0.1:40486)
0806 03:47:20.723180 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0806 03:47:20.723195 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 03:47:20.723250 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:47:20.723421 (+ 171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:47:20.723422 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:47:20.723423 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 03:47:20.723476 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:20.723478 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:20.724091 (+ 613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:20.724093 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:20.724668 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:20.724670 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:20.725039 (+ 369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:20.725040 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:20.725109 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:20.725128 (+ 19us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 03:47:20.725326 (+ 198us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 03:47:20.725337 (+ 11us) client_negotiation.cc:769] Sending connection context
0806 03:47:20.725375 (+ 38us) client_negotiation.cc:240] Negotiation successful
0806 03:47:20.725423 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":51}
I20260806 03:47:20.725503 23772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:20.723135 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40486 (local address 127.11.253.193:33047)
0806 03:47:20.723202 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0806 03:47:20.723204 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:47:20.723265 (+ 61us) server_negotiation.cc:407] Connection header received
0806 03:47:20.723318 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:47:20.723319 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:47:20.723332 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:47:20.723409 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 03:47:20.723541 (+ 132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:20.724007 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:20.724776 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:20.724964 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:20.725041 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:20.725218 (+ 177us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 03:47:20.725255 (+ 37us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 03:47:20.725312 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0806 03:47:20.725389 (+ 77us) server_negotiation.cc:299] Negotiation successful
0806 03:47:20.725411 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260806 03:47:20.726572 23472 raft_consensus.cc:1261] T 379ee64748b24477918b5608950d6d31 P 83f908250a6b4cdc9f27b1141221d4d9 [term 1 FOLLOWER]: Refusing update from remote peer 99f0ce1db669473cb520703bff465d2f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 03:47:20.726565 23675 raft_consensus.cc:1261] T 379ee64748b24477918b5608950d6d31 P 997d20698f9b493eb729667604016e89 [term 1 FOLLOWER]: Refusing update from remote peer 99f0ce1db669473cb520703bff465d2f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 03:47:20.726814 23810 consensus_queue.cc:1035] T 379ee64748b24477918b5608950d6d31 P 99f0ce1db669473cb520703bff465d2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "997d20698f9b493eb729667604016e89" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 37749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 03:47:20.726939 23787 consensus_queue.cc:1035] T 379ee64748b24477918b5608950d6d31 P 99f0ce1db669473cb520703bff465d2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f908250a6b4cdc9f27b1141221d4d9" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 34927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 03:47:20.727437 23788 tablet_replica.cc:439] T 379ee64748b24477918b5608950d6d31 P 83f908250a6b4cdc9f27b1141221d4d9: TxnStatusTablet state changed. Reason: New leader 99f0ce1db669473cb520703bff465d2f. Latest consensus state: current_term: 1 leader_uuid: "99f0ce1db669473cb520703bff465d2f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99f0ce1db669473cb520703bff465d2f" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 33047 } } peers { permanent_uuid: "83f908250a6b4cdc9f27b1141221d4d9" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 34927 } } peers { permanent_uuid: "997d20698f9b493eb729667604016e89" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 37749 } } }
I20260806 03:47:20.727540 23788 tablet_replica.cc:442] T 379ee64748b24477918b5608950d6d31 P 83f908250a6b4cdc9f27b1141221d4d9: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 03:47:20.727484 23786 tablet_replica.cc:439] T 379ee64748b24477918b5608950d6d31 P 997d20698f9b493eb729667604016e89: TxnStatusTablet state changed. Reason: New leader 99f0ce1db669473cb520703bff465d2f. Latest consensus state: current_term: 1 leader_uuid: "99f0ce1db669473cb520703bff465d2f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99f0ce1db669473cb520703bff465d2f" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 33047 } } peers { permanent_uuid: "83f908250a6b4cdc9f27b1141221d4d9" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 34927 } } peers { permanent_uuid: "997d20698f9b493eb729667604016e89" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 37749 } } }
I20260806 03:47:20.727581 23786 tablet_replica.cc:442] T 379ee64748b24477918b5608950d6d31 P 997d20698f9b493eb729667604016e89: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 03:47:20.728165 23810 tablet_replica.cc:439] T 379ee64748b24477918b5608950d6d31 P 99f0ce1db669473cb520703bff465d2f: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "99f0ce1db669473cb520703bff465d2f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99f0ce1db669473cb520703bff465d2f" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 33047 } } peers { permanent_uuid: "83f908250a6b4cdc9f27b1141221d4d9" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 34927 } } peers { permanent_uuid: "997d20698f9b493eb729667604016e89" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 37749 } } }
I20260806 03:47:20.728250 23810 tablet_replica.cc:442] T 379ee64748b24477918b5608950d6d31 P 99f0ce1db669473cb520703bff465d2f: This TxnStatusTablet replica's current role is: LEADER
I20260806 03:47:20.728305 23810 tablet_replica.cc:439] T 379ee64748b24477918b5608950d6d31 P 99f0ce1db669473cb520703bff465d2f: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "99f0ce1db669473cb520703bff465d2f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99f0ce1db669473cb520703bff465d2f" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 33047 } } peers { permanent_uuid: "83f908250a6b4cdc9f27b1141221d4d9" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 34927 } } peers { permanent_uuid: "997d20698f9b493eb729667604016e89" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 37749 } } }
I20260806 03:47:20.728343 23810 tablet_replica.cc:442] T 379ee64748b24477918b5608950d6d31 P 99f0ce1db669473cb520703bff465d2f: This TxnStatusTablet replica's current role is: LEADER
I20260806 03:47:20.728427 23799 mvcc.cc:205] Tried to move back new op lower bound from 7315407014814330880 to 7315407014748585984. Current Snapshot: MvccSnapshot[applied={T|T < 7315407014814330880 or (T in {7315407014814330880})}]
I20260806 03:47:20.728410 23803 mvcc.cc:205] Tried to move back new op lower bound from 7315407014814330880 to 7315407014748585984. Current Snapshot: MvccSnapshot[applied={T|T < 7315407014814330880 or (T in {7315407014814330880})}]
I20260806 03:47:20.728585 23786 tablet_replica.cc:439] T 379ee64748b24477918b5608950d6d31 P 997d20698f9b493eb729667604016e89: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "99f0ce1db669473cb520703bff465d2f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99f0ce1db669473cb520703bff465d2f" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 33047 } } peers { permanent_uuid: "83f908250a6b4cdc9f27b1141221d4d9" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 34927 } } peers { permanent_uuid: "997d20698f9b493eb729667604016e89" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 37749 } } }
I20260806 03:47:20.728657 23786 tablet_replica.cc:442] T 379ee64748b24477918b5608950d6d31 P 997d20698f9b493eb729667604016e89: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 03:47:20.728955 23804 mvcc.cc:205] Tried to move back new op lower bound from 7315407014814330880 to 7315407014748585984. Current Snapshot: MvccSnapshot[applied={T|T < 7315407014814330880}]
I20260806 03:47:20.729048 23788 tablet_replica.cc:439] T 379ee64748b24477918b5608950d6d31 P 83f908250a6b4cdc9f27b1141221d4d9: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "99f0ce1db669473cb520703bff465d2f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99f0ce1db669473cb520703bff465d2f" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 33047 } } peers { permanent_uuid: "83f908250a6b4cdc9f27b1141221d4d9" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 34927 } } peers { permanent_uuid: "997d20698f9b493eb729667604016e89" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 37749 } } }
I20260806 03:47:20.729108 23788 tablet_replica.cc:442] T 379ee64748b24477918b5608950d6d31 P 83f908250a6b4cdc9f27b1141221d4d9: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 03:47:20.735459 23772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:20.732712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.253.193:34515 (local address 127.11.253.193:33047)
0806 03:47:20.732756 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0806 03:47:20.732758 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:47:20.732813 (+ 55us) server_negotiation.cc:407] Connection header received
0806 03:47:20.732871 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:47:20.732872 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:47:20.732880 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:47:20.732930 (+ 50us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 03:47:20.733117 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:20.733564 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:20.734665 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:20.734853 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:20.734915 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:20.735124 (+ 209us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 03:47:20.735154 (+ 30us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 03:47:20.735250 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0806 03:47:20.735359 (+ 109us) server_negotiation.cc:299] Negotiation successful
0806 03:47:20.735377 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260806 03:47:20.735539 23844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:20.732611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.193:33047 (local address 127.11.253.193:34515)
0806 03:47:20.732734 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0806 03:47:20.732746 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 03:47:20.732814 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:47:20.732950 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:47:20.732950 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:47:20.732952 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 03:47:20.732998 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:20.732999 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:20.733645 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:20.733646 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:20.734578 (+ 932us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:20.734581 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:20.734939 (+ 358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:20.734940 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:20.734998 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:20.735015 (+ 17us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 03:47:20.735250 (+ 235us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 03:47:20.735252 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 03:47:20.735354 (+ 102us) client_negotiation.cc:240] Negotiation successful
0806 03:47:20.735408 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":84,"thread_start_us":63,"threads_started":1}
I20260806 03:47:20.738003 23675 raft_consensus.cc:1261] T a1a1c59d5ae242878165796d2c58be06 P 997d20698f9b493eb729667604016e89 [term 1 FOLLOWER]: Refusing update from remote peer 99f0ce1db669473cb520703bff465d2f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 03:47:20.738198 23472 raft_consensus.cc:1261] T a1a1c59d5ae242878165796d2c58be06 P 83f908250a6b4cdc9f27b1141221d4d9 [term 1 FOLLOWER]: Refusing update from remote peer 99f0ce1db669473cb520703bff465d2f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 03:47:20.738302 23787 consensus_queue.cc:1035] T a1a1c59d5ae242878165796d2c58be06 P 99f0ce1db669473cb520703bff465d2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "997d20698f9b493eb729667604016e89" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 37749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 03:47:20.738577 23787 consensus_queue.cc:1035] T a1a1c59d5ae242878165796d2c58be06 P 99f0ce1db669473cb520703bff465d2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f908250a6b4cdc9f27b1141221d4d9" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 34927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 03:47:20.753449 23045 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56072:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 03:47:20.753571 23045 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56072:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 03:47:20.754324 23045 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1faf2e28f8564261b4b003157ec7c205: Sending DeleteTablet for 3 replicas of tablet a1a1c59d5ae242878165796d2c58be06
I20260806 03:47:20.754590 23641 tablet_service.cc:1514] Processing DeleteTablet for tablet a1a1c59d5ae242878165796d2c58be06 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 03:47:20 UTC) from {username='oryx', principal='oryx/127.11.253.254@KRBTEST.COM'} at 127.0.0.1:43088
I20260806 03:47:20.754667 23241 tablet_service.cc:1514] Processing DeleteTablet for tablet a1a1c59d5ae242878165796d2c58be06 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 03:47:20 UTC) from {username='oryx', principal='oryx/127.11.253.254@KRBTEST.COM'} at 127.0.0.1:40468
I20260806 03:47:20.754694 23813 tablet_replica.cc:330] T a1a1c59d5ae242878165796d2c58be06 P 997d20698f9b493eb729667604016e89: stopping tablet replica
I20260806 03:47:20.754745 23813 raft_consensus.cc:2229] T a1a1c59d5ae242878165796d2c58be06 P 997d20698f9b493eb729667604016e89 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 03:47:20.754756 23814 tablet_replica.cc:330] T a1a1c59d5ae242878165796d2c58be06 P 99f0ce1db669473cb520703bff465d2f: stopping tablet replica
I20260806 03:47:20.754784 23813 raft_consensus.cc:2258] T a1a1c59d5ae242878165796d2c58be06 P 997d20698f9b493eb729667604016e89 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 03:47:20.754801 23814 raft_consensus.cc:2229] T a1a1c59d5ae242878165796d2c58be06 P 99f0ce1db669473cb520703bff465d2f [term 1 LEADER]: Raft consensus shutting down.
I20260806 03:47:20.754858 23814 raft_consensus.cc:2258] T a1a1c59d5ae242878165796d2c58be06 P 99f0ce1db669473cb520703bff465d2f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 03:47:20.754966 23438 tablet_service.cc:1514] Processing DeleteTablet for tablet a1a1c59d5ae242878165796d2c58be06 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 03:47:20 UTC) from {username='oryx', principal='oryx/127.11.253.254@KRBTEST.COM'} at 127.0.0.1:57708
I20260806 03:47:20.755066 23815 tablet_replica.cc:330] T a1a1c59d5ae242878165796d2c58be06 P 83f908250a6b4cdc9f27b1141221d4d9: stopping tablet replica
I20260806 03:47:20.755119 23815 raft_consensus.cc:2229] T a1a1c59d5ae242878165796d2c58be06 P 83f908250a6b4cdc9f27b1141221d4d9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 03:47:20.755160 23815 raft_consensus.cc:2258] T a1a1c59d5ae242878165796d2c58be06 P 83f908250a6b4cdc9f27b1141221d4d9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 03:47:20.755278 23814 ts_tablet_manager.cc:1905] T a1a1c59d5ae242878165796d2c58be06 P 99f0ce1db669473cb520703bff465d2f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 03:47:20.755353 23813 ts_tablet_manager.cc:1905] T a1a1c59d5ae242878165796d2c58be06 P 997d20698f9b493eb729667604016e89: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 03:47:20.757186 23813 ts_tablet_manager.cc:1918] T a1a1c59d5ae242878165796d2c58be06 P 997d20698f9b493eb729667604016e89: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 03:47:20.757181 23814 ts_tablet_manager.cc:1918] T a1a1c59d5ae242878165796d2c58be06 P 99f0ce1db669473cb520703bff465d2f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 03:47:20.757251 23813 log.cc:1196] T a1a1c59d5ae242878165796d2c58be06 P 997d20698f9b493eb729667604016e89: Deleting WAL directory at /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785988004637995-12279-0/minicluster-data/ts-2/wal/wals/a1a1c59d5ae242878165796d2c58be06
I20260806 03:47:20.757251 23814 log.cc:1196] T a1a1c59d5ae242878165796d2c58be06 P 99f0ce1db669473cb520703bff465d2f: Deleting WAL directory at /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785988004637995-12279-0/minicluster-data/ts-0/wal/wals/a1a1c59d5ae242878165796d2c58be06
I20260806 03:47:20.757496 23814 ts_tablet_manager.cc:1939] T a1a1c59d5ae242878165796d2c58be06 P 99f0ce1db669473cb520703bff465d2f: Deleting consensus metadata
I20260806 03:47:20.757499 23813 ts_tablet_manager.cc:1939] T a1a1c59d5ae242878165796d2c58be06 P 997d20698f9b493eb729667604016e89: Deleting consensus metadata
I20260806 03:47:20.757892 23031 catalog_manager.cc:4841] TS 997d20698f9b493eb729667604016e89 (127.11.253.195:37749): tablet a1a1c59d5ae242878165796d2c58be06 (table test-table [id=fe6e4106ef614ee296745c9240bc65f1]) successfully deleted
I20260806 03:47:20.758152 23815 ts_tablet_manager.cc:1905] T a1a1c59d5ae242878165796d2c58be06 P 83f908250a6b4cdc9f27b1141221d4d9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 03:47:20.758471 23029 catalog_manager.cc:4841] TS 99f0ce1db669473cb520703bff465d2f (127.11.253.193:33047): tablet a1a1c59d5ae242878165796d2c58be06 (table test-table [id=fe6e4106ef614ee296745c9240bc65f1]) successfully deleted
I20260806 03:47:20.759879 23815 ts_tablet_manager.cc:1918] T a1a1c59d5ae242878165796d2c58be06 P 83f908250a6b4cdc9f27b1141221d4d9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 03:47:20.759958 23815 log.cc:1196] T a1a1c59d5ae242878165796d2c58be06 P 83f908250a6b4cdc9f27b1141221d4d9: Deleting WAL directory at /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785988004637995-12279-0/minicluster-data/ts-1/wal/wals/a1a1c59d5ae242878165796d2c58be06
I20260806 03:47:20.760170 23815 ts_tablet_manager.cc:1939] T a1a1c59d5ae242878165796d2c58be06 P 83f908250a6b4cdc9f27b1141221d4d9: Deleting consensus metadata
I20260806 03:47:20.760672 23029 catalog_manager.cc:4841] TS 83f908250a6b4cdc9f27b1141221d4d9 (127.11.253.194:34927): tablet a1a1c59d5ae242878165796d2c58be06 (table test-table [id=fe6e4106ef614ee296745c9240bc65f1]) successfully deleted
Aug 06 03:47:20 dist-test-slave-6ntk krb5kdc[22990](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988040, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 03:47:20 dist-test-slave-6ntk krb5kdc[22990](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.11.253.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 03:47:20 dist-test-slave-6ntk krb5kdc[22990](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.11.253.254@KRBTEST.COM, Server not found in Kerberos database
W20260806 03:47:20.784579 23127 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 03:47:20.777374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56090 (local address 127.11.253.254:38857)
0806 03:47:20.777452 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0806 03:47:20.777455 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 03:47:20.777598 (+ 143us) server_negotiation.cc:407] Connection header received
0806 03:47:20.777658 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:47:20.777660 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:47:20.777672 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:47:20.777747 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:47:20.778587 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:20.779402 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:20.780451 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:20.780758 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:20.780837 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:20.784489 (+ 3652us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56090: BlockingRecv error: recv got EOF from 127.0.0.1:56090 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
Aug 06 03:47:20 dist-test-slave-6ntk krb5kdc[22990](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988040, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.11.253.254@KRBTEST.COM
I20260806 03:47:20.793035 23127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:20.785231 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56100 (local address 127.11.253.254:38857)
0806 03:47:20.785285 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0806 03:47:20.785287 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:47:20.785381 (+ 94us) server_negotiation.cc:407] Connection header received
0806 03:47:20.785409 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:47:20.785410 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:47:20.785421 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:47:20.785476 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:47:20.786075 (+ 599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:20.786755 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:20.787527 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:20.787798 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:20.787869 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:20.790276 (+ 2407us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:47:20.790278 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:47:20.790278 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:47:20.790286 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:47:20.791834 (+ 1548us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:47:20.792171 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:47:20.792173 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:47:20.792173 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:47:20.792241 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:47:20.792494 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:47:20.792495 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:47:20.792495 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:47:20.792637 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:47:20.792746 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0806 03:47:20.792852 (+ 106us) server_negotiation.cc:299] Negotiation successful
0806 03:47:20.792936 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260806 03:47:20.794729 23045 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56100:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 03:47:20.794916 23045 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 03:47:20.796430 23641 tablet_service.cc:1467] Processing CreateTablet for tablet 1e33da649fd44e27acc3c564f79cbe10 (DEFAULT_TABLE table=test-table [id=6529a67b71e24c8096e9799622d08363]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 03:47:20.796416 23438 tablet_service.cc:1467] Processing CreateTablet for tablet 1e33da649fd44e27acc3c564f79cbe10 (DEFAULT_TABLE table=test-table [id=6529a67b71e24c8096e9799622d08363]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 03:47:20.796437 23241 tablet_service.cc:1467] Processing CreateTablet for tablet 1e33da649fd44e27acc3c564f79cbe10 (DEFAULT_TABLE table=test-table [id=6529a67b71e24c8096e9799622d08363]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 03:47:20.796563 23241 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1e33da649fd44e27acc3c564f79cbe10. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:47:20.796561 23641 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1e33da649fd44e27acc3c564f79cbe10. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:47:20.796561 23438 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1e33da649fd44e27acc3c564f79cbe10. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:47:20.797900 23776 tablet_bootstrap.cc:492] T 1e33da649fd44e27acc3c564f79cbe10 P 99f0ce1db669473cb520703bff465d2f: Bootstrap starting.
I20260806 03:47:20.797912 23779 tablet_bootstrap.cc:492] T 1e33da649fd44e27acc3c564f79cbe10 P 83f908250a6b4cdc9f27b1141221d4d9: Bootstrap starting.
I20260806 03:47:20.798211 23776 tablet_bootstrap.cc:654] T 1e33da649fd44e27acc3c564f79cbe10 P 99f0ce1db669473cb520703bff465d2f: Neither blocks nor log segments found. Creating new log.
I20260806 03:47:20.798334 23779 tablet_bootstrap.cc:654] T 1e33da649fd44e27acc3c564f79cbe10 P 83f908250a6b4cdc9f27b1141221d4d9: Neither blocks nor log segments found. Creating new log.
I20260806 03:47:20.798470 23777 tablet_bootstrap.cc:492] T 1e33da649fd44e27acc3c564f79cbe10 P 997d20698f9b493eb729667604016e89: Bootstrap starting.
I20260806 03:47:20.798838 23777 tablet_bootstrap.cc:654] T 1e33da649fd44e27acc3c564f79cbe10 P 997d20698f9b493eb729667604016e89: Neither blocks nor log segments found. Creating new log.
I20260806 03:47:20.799542 23776 tablet_bootstrap.cc:492] T 1e33da649fd44e27acc3c564f79cbe10 P 99f0ce1db669473cb520703bff465d2f: No bootstrap required, opened a new log
I20260806 03:47:20.799602 23776 ts_tablet_manager.cc:1397] T 1e33da649fd44e27acc3c564f79cbe10 P 99f0ce1db669473cb520703bff465d2f: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260806 03:47:20.799614 23777 tablet_bootstrap.cc:492] T 1e33da649fd44e27acc3c564f79cbe10 P 997d20698f9b493eb729667604016e89: No bootstrap required, opened a new log
I20260806 03:47:20.799669 23777 ts_tablet_manager.cc:1397] T 1e33da649fd44e27acc3c564f79cbe10 P 997d20698f9b493eb729667604016e89: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260806 03:47:20.799707 23776 raft_consensus.cc:348] T 1e33da649fd44e27acc3c564f79cbe10 P 99f0ce1db669473cb520703bff465d2f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99f0ce1db669473cb520703bff465d2f" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 33047 } } peers { permanent_uuid: "83f908250a6b4cdc9f27b1141221d4d9" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 34927 } } peers { permanent_uuid: "997d20698f9b493eb729667604016e89" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 37749 } }
I20260806 03:47:20.799789 23776 raft_consensus.cc:374] T 1e33da649fd44e27acc3c564f79cbe10 P 99f0ce1db669473cb520703bff465d2f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:47:20.799820 23776 raft_consensus.cc:729] T 1e33da649fd44e27acc3c564f79cbe10 P 99f0ce1db669473cb520703bff465d2f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99f0ce1db669473cb520703bff465d2f, State: Initialized, Role: FOLLOWER
I20260806 03:47:20.799865 23776 consensus_queue.cc:260] T 1e33da649fd44e27acc3c564f79cbe10 P 99f0ce1db669473cb520703bff465d2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99f0ce1db669473cb520703bff465d2f" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 33047 } } peers { permanent_uuid: "83f908250a6b4cdc9f27b1141221d4d9" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 34927 } } peers { permanent_uuid: "997d20698f9b493eb729667604016e89" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 37749 } }
I20260806 03:47:20.799852 23777 raft_consensus.cc:348] T 1e33da649fd44e27acc3c564f79cbe10 P 997d20698f9b493eb729667604016e89 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99f0ce1db669473cb520703bff465d2f" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 33047 } } peers { permanent_uuid: "83f908250a6b4cdc9f27b1141221d4d9" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 34927 } } peers { permanent_uuid: "997d20698f9b493eb729667604016e89" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 37749 } }
I20260806 03:47:20.799917 23777 raft_consensus.cc:374] T 1e33da649fd44e27acc3c564f79cbe10 P 997d20698f9b493eb729667604016e89 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:47:20.799942 23777 raft_consensus.cc:729] T 1e33da649fd44e27acc3c564f79cbe10 P 997d20698f9b493eb729667604016e89 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 997d20698f9b493eb729667604016e89, State: Initialized, Role: FOLLOWER
I20260806 03:47:20.799952 23776 ts_tablet_manager.cc:1428] T 1e33da649fd44e27acc3c564f79cbe10 P 99f0ce1db669473cb520703bff465d2f: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260806 03:47:20.799983 23777 consensus_queue.cc:260] T 1e33da649fd44e27acc3c564f79cbe10 P 997d20698f9b493eb729667604016e89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99f0ce1db669473cb520703bff465d2f" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 33047 } } peers { permanent_uuid: "83f908250a6b4cdc9f27b1141221d4d9" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 34927 } } peers { permanent_uuid: "997d20698f9b493eb729667604016e89" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 37749 } }
I20260806 03:47:20.800078 23777 ts_tablet_manager.cc:1428] T 1e33da649fd44e27acc3c564f79cbe10 P 997d20698f9b493eb729667604016e89: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260806 03:47:20.800885 23779 tablet_bootstrap.cc:492] T 1e33da649fd44e27acc3c564f79cbe10 P 83f908250a6b4cdc9f27b1141221d4d9: No bootstrap required, opened a new log
I20260806 03:47:20.800954 23779 ts_tablet_manager.cc:1397] T 1e33da649fd44e27acc3c564f79cbe10 P 83f908250a6b4cdc9f27b1141221d4d9: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20260806 03:47:20.801141 23779 raft_consensus.cc:348] T 1e33da649fd44e27acc3c564f79cbe10 P 83f908250a6b4cdc9f27b1141221d4d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99f0ce1db669473cb520703bff465d2f" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 33047 } } peers { permanent_uuid: "83f908250a6b4cdc9f27b1141221d4d9" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 34927 } } peers { permanent_uuid: "997d20698f9b493eb729667604016e89" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 37749 } }
I20260806 03:47:20.801236 23779 raft_consensus.cc:374] T 1e33da649fd44e27acc3c564f79cbe10 P 83f908250a6b4cdc9f27b1141221d4d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:47:20.801270 23779 raft_consensus.cc:729] T 1e33da649fd44e27acc3c564f79cbe10 P 83f908250a6b4cdc9f27b1141221d4d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83f908250a6b4cdc9f27b1141221d4d9, State: Initialized, Role: FOLLOWER
I20260806 03:47:20.801313 23779 consensus_queue.cc:260] T 1e33da649fd44e27acc3c564f79cbe10 P 83f908250a6b4cdc9f27b1141221d4d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99f0ce1db669473cb520703bff465d2f" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 33047 } } peers { permanent_uuid: "83f908250a6b4cdc9f27b1141221d4d9" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 34927 } } peers { permanent_uuid: "997d20698f9b493eb729667604016e89" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 37749 } }
I20260806 03:47:20.801411 23779 ts_tablet_manager.cc:1428] T 1e33da649fd44e27acc3c564f79cbe10 P 83f908250a6b4cdc9f27b1141221d4d9: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260806 03:47:20.915822 23795 raft_consensus.cc:482] T 1e33da649fd44e27acc3c564f79cbe10 P 99f0ce1db669473cb520703bff465d2f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 03:47:20.915948 23795 raft_consensus.cc:504] T 1e33da649fd44e27acc3c564f79cbe10 P 99f0ce1db669473cb520703bff465d2f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99f0ce1db669473cb520703bff465d2f" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 33047 } } peers { permanent_uuid: "83f908250a6b4cdc9f27b1141221d4d9" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 34927 } } peers { permanent_uuid: "997d20698f9b493eb729667604016e89" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 37749 } }
I20260806 03:47:20.916175 23795 leader_election.cc:288] T 1e33da649fd44e27acc3c564f79cbe10 P 99f0ce1db669473cb520703bff465d2f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 83f908250a6b4cdc9f27b1141221d4d9 (127.11.253.194:34927), 997d20698f9b493eb729667604016e89 (127.11.253.195:37749)
I20260806 03:47:20.916424 23674 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1e33da649fd44e27acc3c564f79cbe10" candidate_uuid: "99f0ce1db669473cb520703bff465d2f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "997d20698f9b493eb729667604016e89" is_pre_election: true
I20260806 03:47:20.916404 23472 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1e33da649fd44e27acc3c564f79cbe10" candidate_uuid: "99f0ce1db669473cb520703bff465d2f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f908250a6b4cdc9f27b1141221d4d9" is_pre_election: true
I20260806 03:47:20.916545 23674 raft_consensus.cc:2454] T 1e33da649fd44e27acc3c564f79cbe10 P 997d20698f9b493eb729667604016e89 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99f0ce1db669473cb520703bff465d2f in term 0.
I20260806 03:47:20.916543 23472 raft_consensus.cc:2454] T 1e33da649fd44e27acc3c564f79cbe10 P 83f908250a6b4cdc9f27b1141221d4d9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99f0ce1db669473cb520703bff465d2f in term 0.
I20260806 03:47:20.916746 23177 leader_election.cc:302] T 1e33da649fd44e27acc3c564f79cbe10 P 99f0ce1db669473cb520703bff465d2f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f908250a6b4cdc9f27b1141221d4d9, 99f0ce1db669473cb520703bff465d2f; no voters:
I20260806 03:47:20.916882 23795 raft_consensus.cc:2781] T 1e33da649fd44e27acc3c564f79cbe10 P 99f0ce1db669473cb520703bff465d2f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 03:47:20.916931 23795 raft_consensus.cc:482] T 1e33da649fd44e27acc3c564f79cbe10 P 99f0ce1db669473cb520703bff465d2f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 03:47:20.916945 23795 raft_consensus.cc:3037] T 1e33da649fd44e27acc3c564f79cbe10 P 99f0ce1db669473cb520703bff465d2f [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:47:20.917675 23795 raft_consensus.cc:504] T 1e33da649fd44e27acc3c564f79cbe10 P 99f0ce1db669473cb520703bff465d2f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99f0ce1db669473cb520703bff465d2f" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 33047 } } peers { permanent_uuid: "83f908250a6b4cdc9f27b1141221d4d9" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 34927 } } peers { permanent_uuid: "997d20698f9b493eb729667604016e89" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 37749 } }
I20260806 03:47:20.917816 23795 leader_election.cc:288] T 1e33da649fd44e27acc3c564f79cbe10 P 99f0ce1db669473cb520703bff465d2f [CANDIDATE]: Term 1 election: Requested vote from peers 83f908250a6b4cdc9f27b1141221d4d9 (127.11.253.194:34927), 997d20698f9b493eb729667604016e89 (127.11.253.195:37749)
I20260806 03:47:20.918051 23472 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1e33da649fd44e27acc3c564f79cbe10" candidate_uuid: "99f0ce1db669473cb520703bff465d2f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f908250a6b4cdc9f27b1141221d4d9"
I20260806 03:47:20.918141 23472 raft_consensus.cc:3037] T 1e33da649fd44e27acc3c564f79cbe10 P 83f908250a6b4cdc9f27b1141221d4d9 [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:47:20.918095 23674 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1e33da649fd44e27acc3c564f79cbe10" candidate_uuid: "99f0ce1db669473cb520703bff465d2f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "997d20698f9b493eb729667604016e89"
I20260806 03:47:20.918186 23674 raft_consensus.cc:3037] T 1e33da649fd44e27acc3c564f79cbe10 P 997d20698f9b493eb729667604016e89 [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:47:20.918936 23472 raft_consensus.cc:2454] T 1e33da649fd44e27acc3c564f79cbe10 P 83f908250a6b4cdc9f27b1141221d4d9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99f0ce1db669473cb520703bff465d2f in term 1.
I20260806 03:47:20.919139 23177 leader_election.cc:302] T 1e33da649fd44e27acc3c564f79cbe10 P 99f0ce1db669473cb520703bff465d2f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f908250a6b4cdc9f27b1141221d4d9, 99f0ce1db669473cb520703bff465d2f; no voters:
I20260806 03:47:20.919267 23795 raft_consensus.cc:2781] T 1e33da649fd44e27acc3c564f79cbe10 P 99f0ce1db669473cb520703bff465d2f [term 1 FOLLOWER]: Leader election won for term 1
I20260806 03:47:20.919358 23795 raft_consensus.cc:686] T 1e33da649fd44e27acc3c564f79cbe10 P 99f0ce1db669473cb520703bff465d2f [term 1 LEADER]: Becoming Leader. State: Replica: 99f0ce1db669473cb520703bff465d2f, State: Running, Role: LEADER
I20260806 03:47:20.919415 23674 raft_consensus.cc:2454] T 1e33da649fd44e27acc3c564f79cbe10 P 997d20698f9b493eb729667604016e89 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99f0ce1db669473cb520703bff465d2f in term 1.
I20260806 03:47:20.919423 23795 consensus_queue.cc:237] T 1e33da649fd44e27acc3c564f79cbe10 P 99f0ce1db669473cb520703bff465d2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99f0ce1db669473cb520703bff465d2f" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 33047 } } peers { permanent_uuid: "83f908250a6b4cdc9f27b1141221d4d9" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 34927 } } peers { permanent_uuid: "997d20698f9b493eb729667604016e89" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 37749 } }
I20260806 03:47:20.920051 23045 catalog_manager.cc:5495] T 1e33da649fd44e27acc3c564f79cbe10 P 99f0ce1db669473cb520703bff465d2f reported cstate change: term changed from 0 to 1, leader changed from <none> to 99f0ce1db669473cb520703bff465d2f (127.11.253.193). New cstate: current_term: 1 leader_uuid: "99f0ce1db669473cb520703bff465d2f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99f0ce1db669473cb520703bff465d2f" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 33047 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "83f908250a6b4cdc9f27b1141221d4d9" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 34927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "997d20698f9b493eb729667604016e89" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 37749 } health_report { overall_health: UNKNOWN } } }
I20260806 03:47:20.948213 23772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:20.945993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40496 (local address 127.11.253.193:33047)
0806 03:47:20.946056 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0806 03:47:20.946060 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 03:47:20.946075 (+ 15us) server_negotiation.cc:407] Connection header received
0806 03:47:20.946136 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:47:20.946137 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:47:20.946146 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:47:20.946219 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 03:47:20.946346 (+ 127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:20.946867 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:20.947633 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:20.947830 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:20.947884 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:20.948012 (+ 128us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 03:47:20.948038 (+ 26us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 03:47:20.948075 (+ 37us) server_negotiation.cc:1035] Waiting for connection context
0806 03:47:20.948135 (+ 60us) server_negotiation.cc:299] Negotiation successful
0806 03:47:20.948152 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260806 03:47:20.949152 23674 raft_consensus.cc:1261] T 1e33da649fd44e27acc3c564f79cbe10 P 997d20698f9b493eb729667604016e89 [term 1 FOLLOWER]: Refusing update from remote peer 99f0ce1db669473cb520703bff465d2f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 03:47:20.949152 23472 raft_consensus.cc:1261] T 1e33da649fd44e27acc3c564f79cbe10 P 83f908250a6b4cdc9f27b1141221d4d9 [term 1 FOLLOWER]: Refusing update from remote peer 99f0ce1db669473cb520703bff465d2f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 03:47:20.949399 23810 consensus_queue.cc:1035] T 1e33da649fd44e27acc3c564f79cbe10 P 99f0ce1db669473cb520703bff465d2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "997d20698f9b493eb729667604016e89" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 37749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 03:47:20.949530 23795 consensus_queue.cc:1035] T 1e33da649fd44e27acc3c564f79cbe10 P 99f0ce1db669473cb520703bff465d2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f908250a6b4cdc9f27b1141221d4d9" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 34927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 03:47:20.952049 23048 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56100:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 03:47:20.952140 23048 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56100:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 03:47:20.952775 23048 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1faf2e28f8564261b4b003157ec7c205: Sending DeleteTablet for 3 replicas of tablet 1e33da649fd44e27acc3c564f79cbe10
I20260806 03:47:20.953013 23241 tablet_service.cc:1514] Processing DeleteTablet for tablet 1e33da649fd44e27acc3c564f79cbe10 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 03:47:20 UTC) from {username='oryx', principal='oryx/127.11.253.254@KRBTEST.COM'} at 127.0.0.1:40468
I20260806 03:47:20.953038 23438 tablet_service.cc:1514] Processing DeleteTablet for tablet 1e33da649fd44e27acc3c564f79cbe10 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 03:47:20 UTC) from {username='oryx', principal='oryx/127.11.253.254@KRBTEST.COM'} at 127.0.0.1:57708
I20260806 03:47:20.953130 23814 tablet_replica.cc:330] T 1e33da649fd44e27acc3c564f79cbe10 P 99f0ce1db669473cb520703bff465d2f: stopping tablet replica
I20260806 03:47:20.953135 23641 tablet_service.cc:1514] Processing DeleteTablet for tablet 1e33da649fd44e27acc3c564f79cbe10 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 03:47:20 UTC) from {username='oryx', principal='oryx/127.11.253.254@KRBTEST.COM'} at 127.0.0.1:43088
I20260806 03:47:20.953207 23814 raft_consensus.cc:2229] T 1e33da649fd44e27acc3c564f79cbe10 P 99f0ce1db669473cb520703bff465d2f [term 1 LEADER]: Raft consensus shutting down.
I20260806 03:47:20.953224 23815 tablet_replica.cc:330] T 1e33da649fd44e27acc3c564f79cbe10 P 83f908250a6b4cdc9f27b1141221d4d9: stopping tablet replica
I20260806 03:47:20.953238 23813 tablet_replica.cc:330] T 1e33da649fd44e27acc3c564f79cbe10 P 997d20698f9b493eb729667604016e89: stopping tablet replica
I20260806 03:47:20.953289 23815 raft_consensus.cc:2229] T 1e33da649fd44e27acc3c564f79cbe10 P 83f908250a6b4cdc9f27b1141221d4d9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 03:47:20.953299 23813 raft_consensus.cc:2229] T 1e33da649fd44e27acc3c564f79cbe10 P 997d20698f9b493eb729667604016e89 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 03:47:20.953318 23814 raft_consensus.cc:2258] T 1e33da649fd44e27acc3c564f79cbe10 P 99f0ce1db669473cb520703bff465d2f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 03:47:20.953338 23815 raft_consensus.cc:2258] T 1e33da649fd44e27acc3c564f79cbe10 P 83f908250a6b4cdc9f27b1141221d4d9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 03:47:20.953351 23813 raft_consensus.cc:2258] T 1e33da649fd44e27acc3c564f79cbe10 P 997d20698f9b493eb729667604016e89 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 03:47:20.953755 23813 ts_tablet_manager.cc:1905] T 1e33da649fd44e27acc3c564f79cbe10 P 997d20698f9b493eb729667604016e89: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 03:47:20.953795 23815 ts_tablet_manager.cc:1905] T 1e33da649fd44e27acc3c564f79cbe10 P 83f908250a6b4cdc9f27b1141221d4d9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 03:47:20.953805 23048 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56100:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 03:47:20.953975 23048 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 03:47:20.955242 23641 tablet_service.cc:1467] Processing CreateTablet for tablet 08720abb29e34302b9fb7164889f39e1 (DEFAULT_TABLE table=test-table [id=f2e3cf8767034bc09452173c176cdee4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 03:47:20.955324 23241 tablet_service.cc:1467] Processing CreateTablet for tablet 08720abb29e34302b9fb7164889f39e1 (DEFAULT_TABLE table=test-table [id=f2e3cf8767034bc09452173c176cdee4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 03:47:20.955366 23641 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 08720abb29e34302b9fb7164889f39e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:47:20.955487 23241 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 08720abb29e34302b9fb7164889f39e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:47:20.955475 23813 ts_tablet_manager.cc:1918] T 1e33da649fd44e27acc3c564f79cbe10 P 997d20698f9b493eb729667604016e89: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 03:47:20.955536 23813 log.cc:1196] T 1e33da649fd44e27acc3c564f79cbe10 P 997d20698f9b493eb729667604016e89: Deleting WAL directory at /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785988004637995-12279-0/minicluster-data/ts-2/wal/wals/1e33da649fd44e27acc3c564f79cbe10
I20260806 03:47:20.955758 23813 ts_tablet_manager.cc:1939] T 1e33da649fd44e27acc3c564f79cbe10 P 997d20698f9b493eb729667604016e89: Deleting consensus metadata
I20260806 03:47:20.956161 23031 catalog_manager.cc:4841] TS 997d20698f9b493eb729667604016e89 (127.11.253.195:37749): tablet 1e33da649fd44e27acc3c564f79cbe10 (table test-table [id=6529a67b71e24c8096e9799622d08363]) successfully deleted
I20260806 03:47:20.956523 23777 tablet_bootstrap.cc:492] T 08720abb29e34302b9fb7164889f39e1 P 997d20698f9b493eb729667604016e89: Bootstrap starting.
I20260806 03:47:20.956866 23776 tablet_bootstrap.cc:492] T 08720abb29e34302b9fb7164889f39e1 P 99f0ce1db669473cb520703bff465d2f: Bootstrap starting.
I20260806 03:47:20.956889 23777 tablet_bootstrap.cc:654] T 08720abb29e34302b9fb7164889f39e1 P 997d20698f9b493eb729667604016e89: Neither blocks nor log segments found. Creating new log.
I20260806 03:47:20.957052 23814 ts_tablet_manager.cc:1905] T 1e33da649fd44e27acc3c564f79cbe10 P 99f0ce1db669473cb520703bff465d2f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 03:47:20.957162 23776 tablet_bootstrap.cc:654] T 08720abb29e34302b9fb7164889f39e1 P 99f0ce1db669473cb520703bff465d2f: Neither blocks nor log segments found. Creating new log.
I20260806 03:47:20.957562 23777 tablet_bootstrap.cc:492] T 08720abb29e34302b9fb7164889f39e1 P 997d20698f9b493eb729667604016e89: No bootstrap required, opened a new log
I20260806 03:47:20.957620 23777 ts_tablet_manager.cc:1397] T 08720abb29e34302b9fb7164889f39e1 P 997d20698f9b493eb729667604016e89: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260806 03:47:20.957701 23776 tablet_bootstrap.cc:492] T 08720abb29e34302b9fb7164889f39e1 P 99f0ce1db669473cb520703bff465d2f: No bootstrap required, opened a new log
I20260806 03:47:20.957753 23776 ts_tablet_manager.cc:1397] T 08720abb29e34302b9fb7164889f39e1 P 99f0ce1db669473cb520703bff465d2f: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260806 03:47:20.957787 23777 raft_consensus.cc:348] T 08720abb29e34302b9fb7164889f39e1 P 997d20698f9b493eb729667604016e89 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99f0ce1db669473cb520703bff465d2f" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 33047 } } peers { permanent_uuid: "83f908250a6b4cdc9f27b1141221d4d9" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 34927 } } peers { permanent_uuid: "997d20698f9b493eb729667604016e89" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 37749 } }
I20260806 03:47:20.957890 23777 raft_consensus.cc:374] T 08720abb29e34302b9fb7164889f39e1 P 997d20698f9b493eb729667604016e89 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:47:20.957888 23776 raft_consensus.cc:348] T 08720abb29e34302b9fb7164889f39e1 P 99f0ce1db669473cb520703bff465d2f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99f0ce1db669473cb520703bff465d2f" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 33047 } } peers { permanent_uuid: "83f908250a6b4cdc9f27b1141221d4d9" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 34927 } } peers { permanent_uuid: "997d20698f9b493eb729667604016e89" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 37749 } }
I20260806 03:47:20.957926 23777 raft_consensus.cc:729] T 08720abb29e34302b9fb7164889f39e1 P 997d20698f9b493eb729667604016e89 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 997d20698f9b493eb729667604016e89, State: Initialized, Role: FOLLOWER
I20260806 03:47:20.957966 23776 raft_consensus.cc:374] T 08720abb29e34302b9fb7164889f39e1 P 99f0ce1db669473cb520703bff465d2f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:47:20.957998 23776 raft_consensus.cc:729] T 08720abb29e34302b9fb7164889f39e1 P 99f0ce1db669473cb520703bff465d2f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99f0ce1db669473cb520703bff465d2f, State: Initialized, Role: FOLLOWER
I20260806 03:47:20.957988 23777 consensus_queue.cc:260] T 08720abb29e34302b9fb7164889f39e1 P 997d20698f9b493eb729667604016e89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99f0ce1db669473cb520703bff465d2f" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 33047 } } peers { permanent_uuid: "83f908250a6b4cdc9f27b1141221d4d9" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 34927 } } peers { permanent_uuid: "997d20698f9b493eb729667604016e89" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 37749 } }
I20260806 03:47:20.958038 23776 consensus_queue.cc:260] T 08720abb29e34302b9fb7164889f39e1 P 99f0ce1db669473cb520703bff465d2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99f0ce1db669473cb520703bff465d2f" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 33047 } } peers { permanent_uuid: "83f908250a6b4cdc9f27b1141221d4d9" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 34927 } } peers { permanent_uuid: "997d20698f9b493eb729667604016e89" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 37749 } }
I20260806 03:47:20.958076 23777 ts_tablet_manager.cc:1428] T 08720abb29e34302b9fb7164889f39e1 P 997d20698f9b493eb729667604016e89: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260806 03:47:20.958144 23776 ts_tablet_manager.cc:1428] T 08720abb29e34302b9fb7164889f39e1 P 99f0ce1db669473cb520703bff465d2f: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 03:47:20.958473 23814 ts_tablet_manager.cc:1918] T 1e33da649fd44e27acc3c564f79cbe10 P 99f0ce1db669473cb520703bff465d2f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 03:47:20.958530 23814 log.cc:1196] T 1e33da649fd44e27acc3c564f79cbe10 P 99f0ce1db669473cb520703bff465d2f: Deleting WAL directory at /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785988004637995-12279-0/minicluster-data/ts-0/wal/wals/1e33da649fd44e27acc3c564f79cbe10
I20260806 03:47:20.958717 23814 ts_tablet_manager.cc:1939] T 1e33da649fd44e27acc3c564f79cbe10 P 99f0ce1db669473cb520703bff465d2f: Deleting consensus metadata
I20260806 03:47:20.959091 23438 tablet_service.cc:1467] Processing CreateTablet for tablet 08720abb29e34302b9fb7164889f39e1 (DEFAULT_TABLE table=test-table [id=f2e3cf8767034bc09452173c176cdee4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 03:47:20.959245 23438 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 08720abb29e34302b9fb7164889f39e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:47:20.959259 23815 ts_tablet_manager.cc:1918] T 1e33da649fd44e27acc3c564f79cbe10 P 83f908250a6b4cdc9f27b1141221d4d9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 03:47:20.959336 23815 log.cc:1196] T 1e33da649fd44e27acc3c564f79cbe10 P 83f908250a6b4cdc9f27b1141221d4d9: Deleting WAL directory at /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785988004637995-12279-0/minicluster-data/ts-1/wal/wals/1e33da649fd44e27acc3c564f79cbe10
I20260806 03:47:20.959554 23815 ts_tablet_manager.cc:1939] T 1e33da649fd44e27acc3c564f79cbe10 P 83f908250a6b4cdc9f27b1141221d4d9: Deleting consensus metadata
I20260806 03:47:20.959878 23029 catalog_manager.cc:4841] TS 83f908250a6b4cdc9f27b1141221d4d9 (127.11.253.194:34927): tablet 1e33da649fd44e27acc3c564f79cbe10 (table test-table [id=6529a67b71e24c8096e9799622d08363]) successfully deleted
I20260806 03:47:20.960129 23029 catalog_manager.cc:4841] TS 99f0ce1db669473cb520703bff465d2f (127.11.253.193:33047): tablet 1e33da649fd44e27acc3c564f79cbe10 (table test-table [id=6529a67b71e24c8096e9799622d08363]) successfully deleted
I20260806 03:47:20.960515 23779 tablet_bootstrap.cc:492] T 08720abb29e34302b9fb7164889f39e1 P 83f908250a6b4cdc9f27b1141221d4d9: Bootstrap starting.
I20260806 03:47:20.961069 23779 tablet_bootstrap.cc:654] T 08720abb29e34302b9fb7164889f39e1 P 83f908250a6b4cdc9f27b1141221d4d9: Neither blocks nor log segments found. Creating new log.
I20260806 03:47:20.961563 23779 tablet_bootstrap.cc:492] T 08720abb29e34302b9fb7164889f39e1 P 83f908250a6b4cdc9f27b1141221d4d9: No bootstrap required, opened a new log
I20260806 03:47:20.961598 23779 ts_tablet_manager.cc:1397] T 08720abb29e34302b9fb7164889f39e1 P 83f908250a6b4cdc9f27b1141221d4d9: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260806 03:47:20.961699 23779 raft_consensus.cc:348] T 08720abb29e34302b9fb7164889f39e1 P 83f908250a6b4cdc9f27b1141221d4d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99f0ce1db669473cb520703bff465d2f" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 33047 } } peers { permanent_uuid: "83f908250a6b4cdc9f27b1141221d4d9" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 34927 } } peers { permanent_uuid: "997d20698f9b493eb729667604016e89" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 37749 } }
I20260806 03:47:20.961817 23779 raft_consensus.cc:374] T 08720abb29e34302b9fb7164889f39e1 P 83f908250a6b4cdc9f27b1141221d4d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:47:20.961840 23779 raft_consensus.cc:729] T 08720abb29e34302b9fb7164889f39e1 P 83f908250a6b4cdc9f27b1141221d4d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83f908250a6b4cdc9f27b1141221d4d9, State: Initialized, Role: FOLLOWER
I20260806 03:47:20.961905 23779 consensus_queue.cc:260] T 08720abb29e34302b9fb7164889f39e1 P 83f908250a6b4cdc9f27b1141221d4d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99f0ce1db669473cb520703bff465d2f" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 33047 } } peers { permanent_uuid: "83f908250a6b4cdc9f27b1141221d4d9" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 34927 } } peers { permanent_uuid: "997d20698f9b493eb729667604016e89" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 37749 } }
I20260806 03:47:20.961997 23779 ts_tablet_manager.cc:1428] T 08720abb29e34302b9fb7164889f39e1 P 83f908250a6b4cdc9f27b1141221d4d9: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 03:47:21.147841 23795 raft_consensus.cc:482] T 08720abb29e34302b9fb7164889f39e1 P 99f0ce1db669473cb520703bff465d2f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 03:47:21.147953 23795 raft_consensus.cc:504] T 08720abb29e34302b9fb7164889f39e1 P 99f0ce1db669473cb520703bff465d2f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99f0ce1db669473cb520703bff465d2f" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 33047 } } peers { permanent_uuid: "83f908250a6b4cdc9f27b1141221d4d9" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 34927 } } peers { permanent_uuid: "997d20698f9b493eb729667604016e89" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 37749 } }
I20260806 03:47:21.148180 23795 leader_election.cc:288] T 08720abb29e34302b9fb7164889f39e1 P 99f0ce1db669473cb520703bff465d2f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 83f908250a6b4cdc9f27b1141221d4d9 (127.11.253.194:34927), 997d20698f9b493eb729667604016e89 (127.11.253.195:37749)
I20260806 03:47:21.148445 23674 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08720abb29e34302b9fb7164889f39e1" candidate_uuid: "99f0ce1db669473cb520703bff465d2f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "997d20698f9b493eb729667604016e89" is_pre_election: true
I20260806 03:47:21.148476 23472 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08720abb29e34302b9fb7164889f39e1" candidate_uuid: "99f0ce1db669473cb520703bff465d2f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f908250a6b4cdc9f27b1141221d4d9" is_pre_election: true
I20260806 03:47:21.148571 23674 raft_consensus.cc:2454] T 08720abb29e34302b9fb7164889f39e1 P 997d20698f9b493eb729667604016e89 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99f0ce1db669473cb520703bff465d2f in term 0.
I20260806 03:47:21.148571 23472 raft_consensus.cc:2454] T 08720abb29e34302b9fb7164889f39e1 P 83f908250a6b4cdc9f27b1141221d4d9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99f0ce1db669473cb520703bff465d2f in term 0.
I20260806 03:47:21.148733 23178 leader_election.cc:302] T 08720abb29e34302b9fb7164889f39e1 P 99f0ce1db669473cb520703bff465d2f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 997d20698f9b493eb729667604016e89, 99f0ce1db669473cb520703bff465d2f; no voters:
I20260806 03:47:21.148880 23795 raft_consensus.cc:2781] T 08720abb29e34302b9fb7164889f39e1 P 99f0ce1db669473cb520703bff465d2f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 03:47:21.148926 23795 raft_consensus.cc:482] T 08720abb29e34302b9fb7164889f39e1 P 99f0ce1db669473cb520703bff465d2f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 03:47:21.148944 23795 raft_consensus.cc:3037] T 08720abb29e34302b9fb7164889f39e1 P 99f0ce1db669473cb520703bff465d2f [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:47:21.149703 23795 raft_consensus.cc:504] T 08720abb29e34302b9fb7164889f39e1 P 99f0ce1db669473cb520703bff465d2f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99f0ce1db669473cb520703bff465d2f" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 33047 } } peers { permanent_uuid: "83f908250a6b4cdc9f27b1141221d4d9" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 34927 } } peers { permanent_uuid: "997d20698f9b493eb729667604016e89" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 37749 } }
I20260806 03:47:21.149909 23795 leader_election.cc:288] T 08720abb29e34302b9fb7164889f39e1 P 99f0ce1db669473cb520703bff465d2f [CANDIDATE]: Term 1 election: Requested vote from peers 83f908250a6b4cdc9f27b1141221d4d9 (127.11.253.194:34927), 997d20698f9b493eb729667604016e89 (127.11.253.195:37749)
I20260806 03:47:21.150058 23472 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08720abb29e34302b9fb7164889f39e1" candidate_uuid: "99f0ce1db669473cb520703bff465d2f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f908250a6b4cdc9f27b1141221d4d9"
I20260806 03:47:21.150095 23674 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08720abb29e34302b9fb7164889f39e1" candidate_uuid: "99f0ce1db669473cb520703bff465d2f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "997d20698f9b493eb729667604016e89"
I20260806 03:47:21.150157 23472 raft_consensus.cc:3037] T 08720abb29e34302b9fb7164889f39e1 P 83f908250a6b4cdc9f27b1141221d4d9 [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:47:21.150185 23674 raft_consensus.cc:3037] T 08720abb29e34302b9fb7164889f39e1 P 997d20698f9b493eb729667604016e89 [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:47:21.150840 23674 raft_consensus.cc:2454] T 08720abb29e34302b9fb7164889f39e1 P 997d20698f9b493eb729667604016e89 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99f0ce1db669473cb520703bff465d2f in term 1.
I20260806 03:47:21.150839 23472 raft_consensus.cc:2454] T 08720abb29e34302b9fb7164889f39e1 P 83f908250a6b4cdc9f27b1141221d4d9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99f0ce1db669473cb520703bff465d2f in term 1.
I20260806 03:47:21.151037 23178 leader_election.cc:302] T 08720abb29e34302b9fb7164889f39e1 P 99f0ce1db669473cb520703bff465d2f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 997d20698f9b493eb729667604016e89, 99f0ce1db669473cb520703bff465d2f; no voters:
I20260806 03:47:21.151156 23795 raft_consensus.cc:2781] T 08720abb29e34302b9fb7164889f39e1 P 99f0ce1db669473cb520703bff465d2f [term 1 FOLLOWER]: Leader election won for term 1
I20260806 03:47:21.151217 23795 raft_consensus.cc:686] T 08720abb29e34302b9fb7164889f39e1 P 99f0ce1db669473cb520703bff465d2f [term 1 LEADER]: Becoming Leader. State: Replica: 99f0ce1db669473cb520703bff465d2f, State: Running, Role: LEADER
I20260806 03:47:21.151279 23795 consensus_queue.cc:237] T 08720abb29e34302b9fb7164889f39e1 P 99f0ce1db669473cb520703bff465d2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99f0ce1db669473cb520703bff465d2f" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 33047 } } peers { permanent_uuid: "83f908250a6b4cdc9f27b1141221d4d9" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 34927 } } peers { permanent_uuid: "997d20698f9b493eb729667604016e89" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 37749 } }
I20260806 03:47:21.151898 23044 catalog_manager.cc:5495] T 08720abb29e34302b9fb7164889f39e1 P 99f0ce1db669473cb520703bff465d2f reported cstate change: term changed from 0 to 1, leader changed from <none> to 99f0ce1db669473cb520703bff465d2f (127.11.253.193). New cstate: current_term: 1 leader_uuid: "99f0ce1db669473cb520703bff465d2f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99f0ce1db669473cb520703bff465d2f" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 33047 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "83f908250a6b4cdc9f27b1141221d4d9" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 34927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "997d20698f9b493eb729667604016e89" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 37749 } health_report { overall_health: UNKNOWN } } }
I20260806 03:47:21.193239 23674 raft_consensus.cc:1261] T 08720abb29e34302b9fb7164889f39e1 P 997d20698f9b493eb729667604016e89 [term 1 FOLLOWER]: Refusing update from remote peer 99f0ce1db669473cb520703bff465d2f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 03:47:21.193239 23472 raft_consensus.cc:1261] T 08720abb29e34302b9fb7164889f39e1 P 83f908250a6b4cdc9f27b1141221d4d9 [term 1 FOLLOWER]: Refusing update from remote peer 99f0ce1db669473cb520703bff465d2f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 03:47:21.193477 23795 consensus_queue.cc:1035] T 08720abb29e34302b9fb7164889f39e1 P 99f0ce1db669473cb520703bff465d2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "997d20698f9b493eb729667604016e89" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 37749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 03:47:21.193608 23810 consensus_queue.cc:1035] T 08720abb29e34302b9fb7164889f39e1 P 99f0ce1db669473cb520703bff465d2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f908250a6b4cdc9f27b1141221d4d9" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 34927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 03:47:21.203472 23044 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56100:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 03:47:21.203600 23044 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56100:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 03:47:21.204252 23044 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1faf2e28f8564261b4b003157ec7c205: Sending DeleteTablet for 3 replicas of tablet 08720abb29e34302b9fb7164889f39e1
I20260806 03:47:21.204516 23641 tablet_service.cc:1514] Processing DeleteTablet for tablet 08720abb29e34302b9fb7164889f39e1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 03:47:21 UTC) from {username='oryx', principal='oryx/127.11.253.254@KRBTEST.COM'} at 127.0.0.1:43088
I20260806 03:47:21.204546 23438 tablet_service.cc:1514] Processing DeleteTablet for tablet 08720abb29e34302b9fb7164889f39e1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 03:47:21 UTC) from {username='oryx', principal='oryx/127.11.253.254@KRBTEST.COM'} at 127.0.0.1:57708
I20260806 03:47:21.204663 23815 tablet_replica.cc:330] T 08720abb29e34302b9fb7164889f39e1 P 83f908250a6b4cdc9f27b1141221d4d9: stopping tablet replica
I20260806 03:47:21.204658 23241 tablet_service.cc:1514] Processing DeleteTablet for tablet 08720abb29e34302b9fb7164889f39e1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 03:47:21 UTC) from {username='oryx', principal='oryx/127.11.253.254@KRBTEST.COM'} at 127.0.0.1:40468
I20260806 03:47:21.204696 12279 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskToVWRe/build/release/bin/kudu with pid 23154
I20260806 03:47:21.204733 23815 raft_consensus.cc:2229] T 08720abb29e34302b9fb7164889f39e1 P 83f908250a6b4cdc9f27b1141221d4d9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 03:47:21.204773 23815 raft_consensus.cc:2258] T 08720abb29e34302b9fb7164889f39e1 P 83f908250a6b4cdc9f27b1141221d4d9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 03:47:21.204900 23814 tablet_replica.cc:330] T 08720abb29e34302b9fb7164889f39e1 P 99f0ce1db669473cb520703bff465d2f: stopping tablet replica
I20260806 03:47:21.204953 23814 raft_consensus.cc:2229] T 08720abb29e34302b9fb7164889f39e1 P 99f0ce1db669473cb520703bff465d2f [term 1 LEADER]: Raft consensus shutting down.
I20260806 03:47:21.205008 23814 raft_consensus.cc:2258] T 08720abb29e34302b9fb7164889f39e1 P 99f0ce1db669473cb520703bff465d2f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 03:47:21.211210 23815 ts_tablet_manager.cc:1905] T 08720abb29e34302b9fb7164889f39e1 P 83f908250a6b4cdc9f27b1141221d4d9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 03:47:21.212639 23815 ts_tablet_manager.cc:1918] T 08720abb29e34302b9fb7164889f39e1 P 83f908250a6b4cdc9f27b1141221d4d9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 03:47:21.212697 23815 log.cc:1196] T 08720abb29e34302b9fb7164889f39e1 P 83f908250a6b4cdc9f27b1141221d4d9: Deleting WAL directory at /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785988004637995-12279-0/minicluster-data/ts-1/wal/wals/08720abb29e34302b9fb7164889f39e1
I20260806 03:47:21.212862 23815 ts_tablet_manager.cc:1939] T 08720abb29e34302b9fb7164889f39e1 P 83f908250a6b4cdc9f27b1141221d4d9: Deleting consensus metadata
I20260806 03:47:21.213377 23029 catalog_manager.cc:4841] TS 83f908250a6b4cdc9f27b1141221d4d9 (127.11.253.194:34927): tablet 08720abb29e34302b9fb7164889f39e1 (table test-table [id=f2e3cf8767034bc09452173c176cdee4]) successfully deleted
I20260806 03:47:21.213994 23813 tablet_replica.cc:330] T 08720abb29e34302b9fb7164889f39e1 P 997d20698f9b493eb729667604016e89: stopping tablet replica
I20260806 03:47:21.214088 23813 raft_consensus.cc:2229] T 08720abb29e34302b9fb7164889f39e1 P 997d20698f9b493eb729667604016e89 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 03:47:21.214143 23813 raft_consensus.cc:2258] T 08720abb29e34302b9fb7164889f39e1 P 997d20698f9b493eb729667604016e89 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 03:47:21.214454 23813 ts_tablet_manager.cc:1905] T 08720abb29e34302b9fb7164889f39e1 P 997d20698f9b493eb729667604016e89: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 03:47:21.215665 23813 ts_tablet_manager.cc:1918] T 08720abb29e34302b9fb7164889f39e1 P 997d20698f9b493eb729667604016e89: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 03:47:21.215750 23813 log.cc:1196] T 08720abb29e34302b9fb7164889f39e1 P 997d20698f9b493eb729667604016e89: Deleting WAL directory at /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785988004637995-12279-0/minicluster-data/ts-2/wal/wals/08720abb29e34302b9fb7164889f39e1
I20260806 03:47:21.215956 23813 ts_tablet_manager.cc:1939] T 08720abb29e34302b9fb7164889f39e1 P 997d20698f9b493eb729667604016e89: Deleting consensus metadata
I20260806 03:47:21.216320 23031 catalog_manager.cc:4841] TS 997d20698f9b493eb729667604016e89 (127.11.253.195:37749): tablet 08720abb29e34302b9fb7164889f39e1 (table test-table [id=f2e3cf8767034bc09452173c176cdee4]) successfully deleted
W20260806 03:47:21.219684 23029 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.253.193:33047 (error 108)
I20260806 03:47:21.220019 12279 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskToVWRe/build/release/bin/kudu with pid 23348
I20260806 03:47:21.228881 23882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:21.220074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.193:33047 (local address 127.0.0.1:40500)
0806 03:47:21.228751 (+ 8677us) negotiation.cc:107] Waiting for socket to connect
0806 03:47:21.228825 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.253.193:33047: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":8647,"thread_start_us":39,"threads_started":1}
W20260806 03:47:21.234540 23029 catalog_manager.cc:4584] TS 99f0ce1db669473cb520703bff465d2f (127.11.253.193:33047): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 08720abb29e34302b9fb7164889f39e1: Network error: Client connection negotiation failed: client connection to 127.11.253.193:33047: connect: Connection refused (error 111)
I20260806 03:47:21.236512 12279 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskToVWRe/build/release/bin/kudu with pid 23545
I20260806 03:47:21.249133 12279 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskToVWRe/build/release/bin/kudu with pid 23007
2026-08-06T03:47:21Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (3346 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785988004637995-12279-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 03:47:21 dist-test-slave-6ntk krb5kdc[23891](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 03:47:21 dist-test-slave-6ntk krb5kdc[23891](info): set up 2 sockets
Aug 06 03:47:21 dist-test-slave-6ntk krb5kdc[23891](info): commencing operation
krb5kdc: starting...
W20260806 03:47:22.494948 12279 mini_kdc.cc:121] Time spent starting KDC: real 1.212s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785988004637995-12279-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785988004637995-12279-0/krb5kdc/test-user.keytab.
Aug 06 03:47:22 dist-test-slave-6ntk krb5kdc[23891](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988042, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T03:47:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T03:47:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.253.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.253.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.253.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.254.keytab.
Entry for principal kudu/127.11.253.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.254.keytab.
I20260806 03:47:22.625067 12279 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskToVWRe/build/release/bin/kudu
/tmp/dist-test-taskToVWRe/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785988004637995-12279-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785988004637995-12279-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785988004637995-12279-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785988004637995-12279-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.253.254:33087
--webserver_interface=127.11.253.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.254.keytab
--principal=kudu/127.11.253.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.253.212:44735
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.253.254:33087
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785988004637995-12279-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785988004637995-12279-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785988004637995-12279-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:47:22.711551 23912 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:47:22.711833 23912 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:47:22.711876 23912 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:47:22.713155 23912 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 03:47:22.713244 23912 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:47:22.713275 23912 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:47:22.713307 23912 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 03:47:22.713340 23912 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 03:47:22.715111 23912 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.253.212:44735
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785988004637995-12279-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785988004637995-12279-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.253.254:33087
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.254.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.253.254:33087
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785988004637995-12279-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.253.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23912
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785988004637995-12279-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:47:22.715410 23912 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:47:22.715644 23912 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:47:22.720724 23919 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:47:22.720758 23921 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:47:22.720785 23912 server_base.cc:1034] running on GCE node
W20260806 03:47:22.720752 23918 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:47:22.721174 23912 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:47:22.721561 23912 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:47:22.722714 23912 hybrid_clock.cc:648] HybridClock initialized: now 1785988042722691 us; error 36 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-06T03:47:22Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1471 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988004637995-12279-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 03:47:22 dist-test-slave-6ntk krb5kdc[23931](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 03:47:22 dist-test-slave-6ntk krb5kdc[23931](info): set up 2 sockets
Aug 06 03:47:22 dist-test-slave-6ntk krb5kdc[23931](info): commencing operation
krb5kdc: starting...
W20260806 03:47:23.992718 12279 mini_kdc.cc:121] Time spent starting KDC: real 1.239s user 0.005s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988004637995-12279-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988004637995-12279-0/krb5kdc/test-user.keytab.
Aug 06 03:47:24 dist-test-slave-6ntk krb5kdc[23931](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988044, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T03:47:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T03:47:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.253.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.253.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.253.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.254.keytab.
Entry for principal kudu/127.11.253.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.254.keytab.
I20260806 03:47:24.121125 12279 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskToVWRe/build/release/bin/kudu
/tmp/dist-test-taskToVWRe/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988004637995-12279-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988004637995-12279-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988004637995-12279-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988004637995-12279-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.253.254:34179
--webserver_interface=127.11.253.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.254.keytab
--principal=kudu/127.11.253.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.253.212:41419
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.253.254:34179
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988004637995-12279-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988004637995-12279-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988004637995-12279-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:47:24.208094 23958 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:47:24.208355 23958 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:47:24.208395 23958 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:47:24.209530 23958 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 03:47:24.209611 23958 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:47:24.209636 23958 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:47:24.209667 23958 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 03:47:24.209697 23958 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 03:47:24.211282 23958 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.253.212:41419
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988004637995-12279-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988004637995-12279-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.253.254:34179
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.254.keytab
--principal=kudu/127.11.253.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.253.254:34179
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988004637995-12279-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.253.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23958
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988004637995-12279-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:47:24.211584 23958 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:47:24.211839 23958 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:47:24.217214 23968 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:47:24.217235 23967 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:47:24.217223 23971 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:47:24.217773 23958 server_base.cc:1034] running on GCE node
I20260806 03:47:24.218104 23958 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:47:24.218358 23958 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:47:24.219528 23958 hybrid_clock.cc:648] HybridClock initialized: now 1785988044219506 us; error 40 us; skew 500 ppm
Aug 06 03:47:24 dist-test-slave-6ntk krb5kdc[23931](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988044, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.253.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:47:24.222148 23958 init.cc:377] Logged in from keytab as kudu/127.11.253.254@KRBTEST.COM (short username kudu)
I20260806 03:47:24.222644 23958 webserver.cc:460] Webserver started at http://127.11.253.254:45713/ using document root <none> and password file <none>
I20260806 03:47:24.222886 23958 fs_manager.cc:359] Metadata directory not provided
I20260806 03:47:24.222959 23958 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:47:24.223102 23958 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:47:24.224198 23958 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988004637995-12279-0/minicluster-data/master-0/data/instance:
uuid: "29a340ef16a049c09755c2af9a0098dc"
format_stamp: "Formatted at 2026-08-06 03:47:24 on dist-test-slave-6ntk"
server_key: "bba2519134dcce2eb4d20f52a19779c8"
server_key_iv: "7f019a3c5aab51c0fa88b2ae29a90c53"
server_key_version: "encryptionkey@0"
I20260806 03:47:24.224582 23958 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988004637995-12279-0/minicluster-data/master-0/wal/instance:
uuid: "29a340ef16a049c09755c2af9a0098dc"
format_stamp: "Formatted at 2026-08-06 03:47:24 on dist-test-slave-6ntk"
server_key: "bba2519134dcce2eb4d20f52a19779c8"
server_key_iv: "7f019a3c5aab51c0fa88b2ae29a90c53"
server_key_version: "encryptionkey@0"
I20260806 03:47:24.226018 23958 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260806 03:47:24.226858 23980 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:47:24.227069 23958 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 03:47:24.227180 23958 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988004637995-12279-0/minicluster-data/master-0/data,/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988004637995-12279-0/minicluster-data/master-0/wal
uuid: "29a340ef16a049c09755c2af9a0098dc"
format_stamp: "Formatted at 2026-08-06 03:47:24 on dist-test-slave-6ntk"
server_key: "bba2519134dcce2eb4d20f52a19779c8"
server_key_iv: "7f019a3c5aab51c0fa88b2ae29a90c53"
server_key_version: "encryptionkey@0"
I20260806 03:47:24.227265 23958 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988004637995-12279-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988004637995-12279-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988004637995-12279-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:47:24.238654 23958 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:47:24.240797 23958 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:47:24.240985 23958 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:47:24.245675 23958 rpc_server.cc:307] RPC server started. Bound to: 127.11.253.254:34179
I20260806 03:47:24.245733 24054 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.253.254:34179 every 8 connection(s)
I20260806 03:47:24.246181 23958 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988004637995-12279-0/minicluster-data/master-0/data/info.pb
I20260806 03:47:24.246942 12279 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskToVWRe/build/release/bin/kudu as pid 23958
I20260806 03:47:24.247078 12279 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988004637995-12279-0/minicluster-data/master-0/wal/instance
I20260806 03:47:24.247288 12279 external_mini_cluster.cc:1428] Setting key 91887bbb1ef6e4049ef825788bbd53e2
I20260806 03:47:24.247390 24055 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:47:24.251012 24055 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 29a340ef16a049c09755c2af9a0098dc: Bootstrap starting.
I20260806 03:47:24.251744 24055 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 29a340ef16a049c09755c2af9a0098dc: Neither blocks nor log segments found. Creating new log.
I20260806 03:47:24.252056 24055 log.cc:824] T 00000000000000000000000000000000 P 29a340ef16a049c09755c2af9a0098dc: Log is configured to *not* fsync() on all Append() calls
I20260806 03:47:24.252782 24055 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 29a340ef16a049c09755c2af9a0098dc: No bootstrap required, opened a new log
Aug 06 03:47:24 dist-test-slave-6ntk krb5kdc[23931](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988044, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.253.254@KRBTEST.COM
I20260806 03:47:24.255296 24055 raft_consensus.cc:348] T 00000000000000000000000000000000 P 29a340ef16a049c09755c2af9a0098dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29a340ef16a049c09755c2af9a0098dc" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 34179 } }
I20260806 03:47:24.255466 24055 raft_consensus.cc:374] T 00000000000000000000000000000000 P 29a340ef16a049c09755c2af9a0098dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:47:24.255491 24055 raft_consensus.cc:729] T 00000000000000000000000000000000 P 29a340ef16a049c09755c2af9a0098dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29a340ef16a049c09755c2af9a0098dc, State: Initialized, Role: FOLLOWER
I20260806 03:47:24.255712 24055 consensus_queue.cc:260] T 00000000000000000000000000000000 P 29a340ef16a049c09755c2af9a0098dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29a340ef16a049c09755c2af9a0098dc" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 34179 } }
I20260806 03:47:24.255817 24055 raft_consensus.cc:388] T 00000000000000000000000000000000 P 29a340ef16a049c09755c2af9a0098dc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 03:47:24.255861 24055 raft_consensus.cc:482] T 00000000000000000000000000000000 P 29a340ef16a049c09755c2af9a0098dc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 03:47:24.255909 24055 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 29a340ef16a049c09755c2af9a0098dc [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:47:24.256690 24055 raft_consensus.cc:504] T 00000000000000000000000000000000 P 29a340ef16a049c09755c2af9a0098dc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29a340ef16a049c09755c2af9a0098dc" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 34179 } }
I20260806 03:47:24.256803 24055 leader_election.cc:302] T 00000000000000000000000000000000 P 29a340ef16a049c09755c2af9a0098dc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 29a340ef16a049c09755c2af9a0098dc; no voters:
I20260806 03:47:24.256994 24055 leader_election.cc:288] T 00000000000000000000000000000000 P 29a340ef16a049c09755c2af9a0098dc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 03:47:24.257062 24061 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 29a340ef16a049c09755c2af9a0098dc [term 1 FOLLOWER]: Leader election won for term 1
I20260806 03:47:24.257251 24061 raft_consensus.cc:686] T 00000000000000000000000000000000 P 29a340ef16a049c09755c2af9a0098dc [term 1 LEADER]: Becoming Leader. State: Replica: 29a340ef16a049c09755c2af9a0098dc, State: Running, Role: LEADER
I20260806 03:47:24.257267 24055 sys_catalog.cc:564] T 00000000000000000000000000000000 P 29a340ef16a049c09755c2af9a0098dc [sys.catalog]: configured and running, proceeding with master startup.
I20260806 03:47:24.257347 24061 consensus_queue.cc:237] T 00000000000000000000000000000000 P 29a340ef16a049c09755c2af9a0098dc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29a340ef16a049c09755c2af9a0098dc" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 34179 } }
I20260806 03:47:24.257714 24062 sys_catalog.cc:455] T 00000000000000000000000000000000 P 29a340ef16a049c09755c2af9a0098dc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "29a340ef16a049c09755c2af9a0098dc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29a340ef16a049c09755c2af9a0098dc" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 34179 } } }
I20260806 03:47:24.257696 24061 sys_catalog.cc:455] T 00000000000000000000000000000000 P 29a340ef16a049c09755c2af9a0098dc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 29a340ef16a049c09755c2af9a0098dc. Latest consensus state: current_term: 1 leader_uuid: "29a340ef16a049c09755c2af9a0098dc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29a340ef16a049c09755c2af9a0098dc" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 34179 } } }
I20260806 03:47:24.257881 24062 sys_catalog.cc:458] T 00000000000000000000000000000000 P 29a340ef16a049c09755c2af9a0098dc [sys.catalog]: This master's current role is: LEADER
I20260806 03:47:24.257885 24061 sys_catalog.cc:458] T 00000000000000000000000000000000 P 29a340ef16a049c09755c2af9a0098dc [sys.catalog]: This master's current role is: LEADER
I20260806 03:47:24.258147 24073 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 03:47:24.258730 24073 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 03:47:24.259366 24059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:24.247866 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33646 (local address 127.11.253.254:34179)
0806 03:47:24.248070 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0806 03:47:24.248079 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 03:47:24.248113 (+ 34us) server_negotiation.cc:407] Connection header received
0806 03:47:24.248622 (+ 509us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:47:24.248629 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:47:24.248675 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:47:24.248790 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:47:24.249624 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:24.250625 (+ 1001us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:24.251648 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:24.252119 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:24.252225 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:24.255082 (+ 2857us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:47:24.255086 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:47:24.255086 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:47:24.255097 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:47:24.257175 (+ 2078us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:47:24.257618 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:47:24.257621 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:47:24.257622 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:47:24.257733 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:47:24.257996 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:47:24.257997 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:47:24.257997 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:47:24.258333 (+ 336us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:47:24.258519 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0806 03:47:24.259094 (+ 575us) server_negotiation.cc:299] Negotiation successful
0806 03:47:24.259213 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":116,"threads_started":1}
I20260806 03:47:24.260502 24073 catalog_manager.cc:1347] Generated new cluster ID: 4ca64d742e8e46c8be030a36bcb6fa62
I20260806 03:47:24.260561 24073 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 03:47:24.272244 24073 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 03:47:24.273001 24073 catalog_manager.cc:1504] Loading token signing keys...
I20260806 03:47:24.293025 24073 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 29a340ef16a049c09755c2af9a0098dc: Generated new TSK 0
I20260806 03:47:24.293252 24073 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.253.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.253.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.253.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.193.keytab.
Entry for principal kudu/127.11.253.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.193.keytab.
I20260806 03:47:24.327226 12279 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskToVWRe/build/release/bin/kudu
/tmp/dist-test-taskToVWRe/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988004637995-12279-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988004637995-12279-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988004637995-12279-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988004637995-12279-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.253.193:0
--local_ip_for_outbound_sockets=127.11.253.193
--webserver_interface=127.11.253.193
--webserver_port=0
--tserver_master_addrs=127.11.253.254:34179
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.193.keytab
--principal=kudu/127.11.253.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.253.212:41419
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988004637995-12279-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988004637995-12279-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988004637995-12279-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:47:24.413640 24090 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:47:24.413923 24090 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:47:24.413981 24090 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:47:24.415153 24090 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:47:24.415234 24090 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:47:24.415283 24090 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.253.193
I20260806 03:47:24.416769 24090 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.253.212:41419
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988004637995-12279-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988004637995-12279-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.193.keytab
--principal=kudu/127.11.253.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.253.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988004637995-12279-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.253.193
--webserver_port=0
--tserver_master_addrs=127.11.253.254:34179
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24090
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.253.193
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988004637995-12279-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:47:24.417044 24090 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:47:24.417296 24090 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:47:24.422710 24098 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:47:24.422700 24099 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:47:24.422943 24090 server_base.cc:1034] running on GCE node
W20260806 03:47:24.422668 24101 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:47:24.423278 24090 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:47:24.423542 24090 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:47:24.424677 24090 hybrid_clock.cc:648] HybridClock initialized: now 1785988044424661 us; error 28 us; skew 500 ppm
Aug 06 03:47:24 dist-test-slave-6ntk krb5kdc[23931](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988044, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.253.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:47:24.427520 24090 init.cc:377] Logged in from keytab as kudu/127.11.253.193@KRBTEST.COM (short username kudu)
I20260806 03:47:24.428013 24090 webserver.cc:460] Webserver started at http://127.11.253.193:46095/ using document root <none> and password file <none>
I20260806 03:47:24.428246 24090 fs_manager.cc:359] Metadata directory not provided
I20260806 03:47:24.428304 24090 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:47:24.428442 24090 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:47:24.429557 24090 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988004637995-12279-0/minicluster-data/ts-0/data/instance:
uuid: "7c2c120f6cd048798f67ee612082b30e"
format_stamp: "Formatted at 2026-08-06 03:47:24 on dist-test-slave-6ntk"
server_key: "53edd8b0540505de18d90992e8348bd1"
server_key_iv: "4754f07e99c8fca2327971f96f012773"
server_key_version: "encryptionkey@0"
I20260806 03:47:24.429986 24090 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988004637995-12279-0/minicluster-data/ts-0/wal/instance:
uuid: "7c2c120f6cd048798f67ee612082b30e"
format_stamp: "Formatted at 2026-08-06 03:47:24 on dist-test-slave-6ntk"
server_key: "53edd8b0540505de18d90992e8348bd1"
server_key_iv: "4754f07e99c8fca2327971f96f012773"
server_key_version: "encryptionkey@0"
I20260806 03:47:24.431432 24090 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 03:47:24.432195 24112 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:47:24.432521 24090 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 03:47:24.432637 24090 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988004637995-12279-0/minicluster-data/ts-0/data,/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988004637995-12279-0/minicluster-data/ts-0/wal
uuid: "7c2c120f6cd048798f67ee612082b30e"
format_stamp: "Formatted at 2026-08-06 03:47:24 on dist-test-slave-6ntk"
server_key: "53edd8b0540505de18d90992e8348bd1"
server_key_iv: "4754f07e99c8fca2327971f96f012773"
server_key_version: "encryptionkey@0"
I20260806 03:47:24.432714 24090 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988004637995-12279-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988004637995-12279-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988004637995-12279-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:47:24.443729 24090 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:47:24.445793 24090 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:47:24.446041 24090 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:47:24.446300 24090 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 03:47:24.446659 24090 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 03:47:24.446696 24090 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 03:47:24.446719 24090 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 03:47:24.446768 24090 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 03:47:24.452958 24090 rpc_server.cc:307] RPC server started. Bound to: 127.11.253.193:44747
I20260806 03:47:24.453060 24269 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.253.193:44747 every 8 connection(s)
I20260806 03:47:24.453399 24090 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988004637995-12279-0/minicluster-data/ts-0/data/info.pb
Aug 06 03:47:24 dist-test-slave-6ntk krb5kdc[23931](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988044, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.253.193@KRBTEST.COM for kudu/127.11.253.254@KRBTEST.COM
I20260806 03:47:24.462970 24059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:24.454222 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.253.193:46303 (local address 127.11.253.254:34179)
0806 03:47:24.454297 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0806 03:47:24.454299 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:47:24.454500 (+ 201us) server_negotiation.cc:407] Connection header received
0806 03:47:24.454974 (+ 474us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:47:24.454975 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:47:24.454984 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:47:24.455044 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:47:24.456033 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:24.456539 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:24.457615 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:24.457825 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:24.457905 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:24.460097 (+ 2192us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:47:24.460098 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:47:24.460098 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:47:24.460105 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:47:24.461620 (+ 1515us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:47:24.462125 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:47:24.462126 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:47:24.462126 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:47:24.462180 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:47:24.462416 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:47:24.462417 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:47:24.462417 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:47:24.462545 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:47:24.462599 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0806 03:47:24.462835 (+ 236us) server_negotiation.cc:299] Negotiation successful
0806 03:47:24.462903 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260806 03:47:24.463167 24273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:24.454190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:34179 (local address 127.11.253.193:46303)
0806 03:47:24.454352 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0806 03:47:24.454396 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0806 03:47:24.454882 (+ 486us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:47:24.455096 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:47:24.455100 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:47:24.455688 (+ 588us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:47:24.455922 (+ 234us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:24.455928 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:24.456625 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:24.456626 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:24.457535 (+ 909us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:24.457537 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:24.457957 (+ 420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:24.457958 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:24.458108 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:24.458604 (+ 496us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:47:24.458612 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:47:24.460004 (+ 1392us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:47:24.461692 (+ 1688us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:47:24.461699 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:47:24.461701 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:47:24.462034 (+ 333us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:47:24.462245 (+ 211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:47:24.462246 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:47:24.462246 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:47:24.462353 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:47:24.462608 (+ 255us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:47:24.462613 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:47:24.462724 (+ 111us) client_negotiation.cc:769] Sending connection context
0806 03:47:24.462806 (+ 82us) client_negotiation.cc:240] Negotiation successful
0806 03:47:24.462927 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":87,"threads_started":1}
I20260806 03:47:24.463488 12279 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskToVWRe/build/release/bin/kudu as pid 24090
I20260806 03:47:24.463618 12279 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988004637995-12279-0/minicluster-data/ts-0/wal/instance
I20260806 03:47:24.463791 24270 heartbeater.cc:347] Connected to a master server at 127.11.253.254:34179
I20260806 03:47:24.463884 24270 heartbeater.cc:464] Registering TS with master...
I20260806 03:47:24.463904 12279 external_mini_cluster.cc:1428] Setting key 79c7f29a7e2f2ff432f323b8c21ea1fb
I20260806 03:47:24.464115 24270 heartbeater.cc:511] Master 127.11.253.254:34179 requested a full tablet report, sending...
I20260806 03:47:24.464736 24001 ts_manager.cc:194] Registered new tserver with Master: 7c2c120f6cd048798f67ee612082b30e (127.11.253.193:44747)
I20260806 03:47:24.465368 24001 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.253.193@KRBTEST.COM'} at 127.11.253.193:46303
WARNING: no policy specified for kudu/127.11.253.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.253.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.253.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.194.keytab.
Entry for principal kudu/127.11.253.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.194.keytab.
I20260806 03:47:24.493662 12279 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskToVWRe/build/release/bin/kudu
/tmp/dist-test-taskToVWRe/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988004637995-12279-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988004637995-12279-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988004637995-12279-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988004637995-12279-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.253.194:0
--local_ip_for_outbound_sockets=127.11.253.194
--webserver_interface=127.11.253.194
--webserver_port=0
--tserver_master_addrs=127.11.253.254:34179
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.194.keytab
--principal=kudu/127.11.253.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.253.212:41419
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988004637995-12279-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988004637995-12279-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988004637995-12279-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:47:24.578955 24279 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:47:24.579219 24279 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:47:24.579257 24279 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:47:24.580423 24279 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:47:24.580503 24279 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:47:24.580546 24279 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.253.194
I20260806 03:47:24.582013 24279 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.253.212:41419
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988004637995-12279-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988004637995-12279-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.194.keytab
--principal=kudu/127.11.253.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.253.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988004637995-12279-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.253.194
--webserver_port=0
--tserver_master_addrs=127.11.253.254:34179
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24279
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.253.194
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988004637995-12279-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:47:24.582298 24279 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:47:24.582541 24279 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:47:24.588178 24295 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:47:24.588282 24279 server_base.cc:1034] running on GCE node
W20260806 03:47:24.588223 24288 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:47:24.588189 24291 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:47:24.588670 24279 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:47:24.588989 24279 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:47:24.590152 24279 hybrid_clock.cc:648] HybridClock initialized: now 1785988044590140 us; error 30 us; skew 500 ppm
Aug 06 03:47:24 dist-test-slave-6ntk krb5kdc[23931](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988044, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.253.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:47:24.592839 24279 init.cc:377] Logged in from keytab as kudu/127.11.253.194@KRBTEST.COM (short username kudu)
I20260806 03:47:24.593267 24279 webserver.cc:460] Webserver started at http://127.11.253.194:35353/ using document root <none> and password file <none>
I20260806 03:47:24.593458 24279 fs_manager.cc:359] Metadata directory not provided
I20260806 03:47:24.593499 24279 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:47:24.593591 24279 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:47:24.594704 24279 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988004637995-12279-0/minicluster-data/ts-1/data/instance:
uuid: "6aeb2de5b6864b51abf7d0064007dbec"
format_stamp: "Formatted at 2026-08-06 03:47:24 on dist-test-slave-6ntk"
server_key: "99f272b74fe9cbea9c9d9c9ee58288a5"
server_key_iv: "39dc10b3036eb141e9f79906ae3a3b2a"
server_key_version: "encryptionkey@0"
I20260806 03:47:24.595021 24279 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988004637995-12279-0/minicluster-data/ts-1/wal/instance:
uuid: "6aeb2de5b6864b51abf7d0064007dbec"
format_stamp: "Formatted at 2026-08-06 03:47:24 on dist-test-slave-6ntk"
server_key: "99f272b74fe9cbea9c9d9c9ee58288a5"
server_key_iv: "39dc10b3036eb141e9f79906ae3a3b2a"
server_key_version: "encryptionkey@0"
I20260806 03:47:24.596323 24279 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 03:47:24.597108 24304 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:47:24.597290 24279 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 03:47:24.597411 24279 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988004637995-12279-0/minicluster-data/ts-1/data,/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988004637995-12279-0/minicluster-data/ts-1/wal
uuid: "6aeb2de5b6864b51abf7d0064007dbec"
format_stamp: "Formatted at 2026-08-06 03:47:24 on dist-test-slave-6ntk"
server_key: "99f272b74fe9cbea9c9d9c9ee58288a5"
server_key_iv: "39dc10b3036eb141e9f79906ae3a3b2a"
server_key_version: "encryptionkey@0"
I20260806 03:47:24.597492 24279 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988004637995-12279-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988004637995-12279-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988004637995-12279-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:47:24.605572 24279 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:47:24.607968 24279 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:47:24.608134 24279 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:47:24.608376 24279 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 03:47:24.608686 24279 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 03:47:24.608744 24279 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 03:47:24.608785 24279 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 03:47:24.608815 24279 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 03:47:24.614833 24279 rpc_server.cc:307] RPC server started. Bound to: 127.11.253.194:41175
I20260806 03:47:24.614938 24463 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.253.194:41175 every 8 connection(s)
I20260806 03:47:24.615232 24279 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988004637995-12279-0/minicluster-data/ts-1/data/info.pb
I20260806 03:47:24.619545 12279 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskToVWRe/build/release/bin/kudu as pid 24279
I20260806 03:47:24.619661 12279 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988004637995-12279-0/minicluster-data/ts-1/wal/instance
I20260806 03:47:24.619874 12279 external_mini_cluster.cc:1428] Setting key b3d8589d65c3e1c0b6b7b6b4cfa8a28f
Aug 06 03:47:24 dist-test-slave-6ntk krb5kdc[23931](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988044, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.253.194@KRBTEST.COM for kudu/127.11.253.254@KRBTEST.COM
I20260806 03:47:24.626144 24059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:24.616049 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.253.194:59921 (local address 127.11.253.254:34179)
0806 03:47:24.616123 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0806 03:47:24.616126 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 03:47:24.616354 (+ 228us) server_negotiation.cc:407] Connection header received
0806 03:47:24.616830 (+ 476us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:47:24.616832 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:47:24.616844 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:47:24.616910 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:47:24.617945 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:24.618641 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:24.619974 (+ 1333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:24.620245 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:24.620284 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:24.622875 (+ 2591us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:47:24.622876 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:47:24.622877 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:47:24.622885 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:47:24.624368 (+ 1483us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:47:24.624816 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:47:24.624817 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:47:24.624818 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:47:24.624880 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:47:24.625332 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:47:24.625333 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:47:24.625333 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:47:24.625480 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:47:24.625592 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0806 03:47:24.625945 (+ 353us) server_negotiation.cc:299] Negotiation successful
0806 03:47:24.626039 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260806 03:47:24.626374 24467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:24.615997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:34179 (local address 127.11.253.194:59921)
0806 03:47:24.616168 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0806 03:47:24.616213 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 03:47:24.616716 (+ 503us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:47:24.616994 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:47:24.616997 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:47:24.617586 (+ 589us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:47:24.617783 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:24.617788 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:24.618740 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:24.618741 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:24.619856 (+ 1115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:24.619859 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:24.620417 (+ 558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:24.620418 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:24.620561 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:24.621199 (+ 638us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:47:24.621208 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:47:24.622761 (+ 1553us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:47:24.624477 (+ 1716us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:47:24.624485 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:47:24.624487 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:47:24.624723 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:47:24.624959 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:47:24.624960 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:47:24.624961 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:47:24.625082 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:47:24.625580 (+ 498us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:47:24.625588 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:47:24.625747 (+ 159us) client_negotiation.cc:769] Sending connection context
0806 03:47:24.625942 (+ 195us) client_negotiation.cc:240] Negotiation successful
0806 03:47:24.626122 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":94,"threads_started":1}
I20260806 03:47:24.627034 24464 heartbeater.cc:347] Connected to a master server at 127.11.253.254:34179
I20260806 03:47:24.627151 24464 heartbeater.cc:464] Registering TS with master...
I20260806 03:47:24.627339 24464 heartbeater.cc:511] Master 127.11.253.254:34179 requested a full tablet report, sending...
I20260806 03:47:24.627822 24001 ts_manager.cc:194] Registered new tserver with Master: 6aeb2de5b6864b51abf7d0064007dbec (127.11.253.194:41175)
I20260806 03:47:24.628355 24001 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.253.194@KRBTEST.COM'} at 127.11.253.194:59921
WARNING: no policy specified for kudu/127.11.253.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.253.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.253.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.195.keytab.
Entry for principal kudu/127.11.253.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.195.keytab.
I20260806 03:47:24.649714 12279 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskToVWRe/build/release/bin/kudu
/tmp/dist-test-taskToVWRe/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988004637995-12279-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988004637995-12279-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988004637995-12279-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988004637995-12279-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.253.195:0
--local_ip_for_outbound_sockets=127.11.253.195
--webserver_interface=127.11.253.195
--webserver_port=0
--tserver_master_addrs=127.11.253.254:34179
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.195.keytab
--principal=kudu/127.11.253.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.253.212:41419
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988004637995-12279-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988004637995-12279-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988004637995-12279-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:47:24.735414 24471 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:47:24.735615 24471 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:47:24.735637 24471 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:47:24.736778 24471 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:47:24.736821 24471 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:47:24.736853 24471 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.253.195
I20260806 03:47:24.738300 24471 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.253.212:41419
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988004637995-12279-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988004637995-12279-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.195.keytab
--principal=kudu/127.11.253.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.253.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988004637995-12279-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.253.195
--webserver_port=0
--tserver_master_addrs=127.11.253.254:34179
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24471
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.253.195
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988004637995-12279-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:47:24.738499 24471 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:47:24.738689 24471 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:47:24.743948 24481 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:47:24.743973 24478 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:47:24.744262 24479 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:47:24.744339 24471 server_base.cc:1034] running on GCE node
I20260806 03:47:24.744549 24471 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:47:24.744796 24471 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:47:24.745954 24471 hybrid_clock.cc:648] HybridClock initialized: now 1785988044745924 us; error 44 us; skew 500 ppm
Aug 06 03:47:24 dist-test-slave-6ntk krb5kdc[23931](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988044, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.253.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:47:24.748720 24471 init.cc:377] Logged in from keytab as kudu/127.11.253.195@KRBTEST.COM (short username kudu)
I20260806 03:47:24.749177 24471 webserver.cc:460] Webserver started at http://127.11.253.195:33237/ using document root <none> and password file <none>
I20260806 03:47:24.749495 24471 fs_manager.cc:359] Metadata directory not provided
I20260806 03:47:24.749570 24471 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:47:24.749729 24471 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:47:24.751053 24471 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988004637995-12279-0/minicluster-data/ts-2/data/instance:
uuid: "a1b25c1ae9b24bf894980f3e84b6641d"
format_stamp: "Formatted at 2026-08-06 03:47:24 on dist-test-slave-6ntk"
server_key: "33a5f843dab1cccdfec8ff5713387529"
server_key_iv: "423675b2cfe9f88b332f37a977ec7170"
server_key_version: "encryptionkey@0"
I20260806 03:47:24.751380 24471 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988004637995-12279-0/minicluster-data/ts-2/wal/instance:
uuid: "a1b25c1ae9b24bf894980f3e84b6641d"
format_stamp: "Formatted at 2026-08-06 03:47:24 on dist-test-slave-6ntk"
server_key: "33a5f843dab1cccdfec8ff5713387529"
server_key_iv: "423675b2cfe9f88b332f37a977ec7170"
server_key_version: "encryptionkey@0"
I20260806 03:47:24.752708 24471 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 03:47:24.753471 24491 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:47:24.753630 24471 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 03:47:24.753724 24471 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988004637995-12279-0/minicluster-data/ts-2/data,/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988004637995-12279-0/minicluster-data/ts-2/wal
uuid: "a1b25c1ae9b24bf894980f3e84b6641d"
format_stamp: "Formatted at 2026-08-06 03:47:24 on dist-test-slave-6ntk"
server_key: "33a5f843dab1cccdfec8ff5713387529"
server_key_iv: "423675b2cfe9f88b332f37a977ec7170"
server_key_version: "encryptionkey@0"
I20260806 03:47:24.753785 24471 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988004637995-12279-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988004637995-12279-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988004637995-12279-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:47:24.766140 24471 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:47:24.768038 24471 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:47:24.768153 24471 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:47:24.768338 24471 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 03:47:24.768688 24471 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 03:47:24.768721 24471 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 03:47:24.768770 24471 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 03:47:24.768798 24471 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 03:47:24.774665 24471 rpc_server.cc:307] RPC server started. Bound to: 127.11.253.195:40465
I20260806 03:47:24.774741 24655 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.253.195:40465 every 8 connection(s)
I20260806 03:47:24.775120 24471 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988004637995-12279-0/minicluster-data/ts-2/data/info.pb
I20260806 03:47:24.775255 12279 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskToVWRe/build/release/bin/kudu as pid 24471
I20260806 03:47:24.775365 12279 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988004637995-12279-0/minicluster-data/ts-2/wal/instance
I20260806 03:47:24.775596 12279 external_mini_cluster.cc:1428] Setting key 198fd269f09be6e7d4e2d57d39125f03
Aug 06 03:47:24 dist-test-slave-6ntk krb5kdc[23931](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988044, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.253.195@KRBTEST.COM for kudu/127.11.253.254@KRBTEST.COM
I20260806 03:47:24.784600 24059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:24.775947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.253.195:59107 (local address 127.11.253.254:34179)
0806 03:47:24.775999 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0806 03:47:24.776002 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 03:47:24.776352 (+ 350us) server_negotiation.cc:407] Connection header received
0806 03:47:24.776933 (+ 581us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:47:24.776934 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:47:24.776943 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:47:24.776990 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:47:24.778005 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:24.778589 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:24.779581 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:24.779775 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:24.779835 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:24.782110 (+ 2275us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:47:24.782111 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:47:24.782112 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:47:24.782118 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:47:24.783304 (+ 1186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:47:24.783700 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:47:24.783700 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:47:24.783701 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:47:24.783752 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:47:24.783968 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:47:24.783969 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:47:24.783969 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:47:24.784068 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:47:24.784120 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0806 03:47:24.784428 (+ 308us) server_negotiation.cc:299] Negotiation successful
0806 03:47:24.784514 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260806 03:47:24.784663 24659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:24.775928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:34179 (local address 127.11.253.195:59107)
0806 03:47:24.776100 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0806 03:47:24.776226 (+ 126us) client_negotiation.cc:174] Beginning negotiation
0806 03:47:24.776842 (+ 616us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:47:24.777045 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:47:24.777048 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:47:24.777618 (+ 570us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:47:24.777812 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:24.777816 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:24.778683 (+ 867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:24.778683 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:24.779491 (+ 808us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:24.779493 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:24.779863 (+ 370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:24.779864 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:24.779972 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:24.780490 (+ 518us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:47:24.780498 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:47:24.782026 (+ 1528us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:47:24.783387 (+ 1361us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:47:24.783394 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:47:24.783396 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:47:24.783631 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:47:24.783813 (+ 182us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:47:24.783813 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:47:24.783813 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:47:24.783912 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:47:24.784124 (+ 212us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:47:24.784128 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:47:24.784241 (+ 113us) client_negotiation.cc:769] Sending connection context
0806 03:47:24.784368 (+ 127us) client_negotiation.cc:240] Negotiation successful
0806 03:47:24.784491 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":91,"threads_started":1}
I20260806 03:47:24.785194 24656 heartbeater.cc:347] Connected to a master server at 127.11.253.254:34179
I20260806 03:47:24.785305 24656 heartbeater.cc:464] Registering TS with master...
I20260806 03:47:24.785487 24656 heartbeater.cc:511] Master 127.11.253.254:34179 requested a full tablet report, sending...
I20260806 03:47:24.785913 24001 ts_manager.cc:194] Registered new tserver with Master: a1b25c1ae9b24bf894980f3e84b6641d (127.11.253.195:40465)
I20260806 03:47:24.786338 24001 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.253.195@KRBTEST.COM'} at 127.11.253.195:59107
I20260806 03:47:24.787472 12279 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.11.253.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.11.253.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.11.253.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988004637995-12279-0/krb5kdc/oryx_127.11.253.193.keytab.
Entry for principal oryx/127.11.253.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988004637995-12279-0/krb5kdc/oryx_127.11.253.193.keytab.
WARNING: no policy specified for oryx/127.11.253.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.11.253.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.11.253.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988004637995-12279-0/krb5kdc/oryx_127.11.253.194.keytab.
Entry for principal oryx/127.11.253.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988004637995-12279-0/krb5kdc/oryx_127.11.253.194.keytab.
WARNING: no policy specified for oryx/127.11.253.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.11.253.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.11.253.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988004637995-12279-0/krb5kdc/oryx_127.11.253.195.keytab.
Entry for principal oryx/127.11.253.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988004637995-12279-0/krb5kdc/oryx_127.11.253.195.keytab.
I20260806 03:47:24.872197 12279 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskToVWRe/build/release/bin/kudu with pid 24090
I20260806 03:47:24.883812 12279 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskToVWRe/build/release/bin/kudu with pid 24279
I20260806 03:47:24.897315 12279 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskToVWRe/build/release/bin/kudu with pid 24471
I20260806 03:47:24.910571 12279 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskToVWRe/build/release/bin/kudu with pid 23958
I20260806 03:47:24.923286 12279 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskToVWRe/build/release/bin/kudu
/tmp/dist-test-taskToVWRe/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988004637995-12279-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988004637995-12279-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988004637995-12279-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988004637995-12279-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.253.254:34179
--webserver_interface=127.11.253.254
--webserver_port=45713
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.254.keytab
--principal=kudu/127.11.253.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.253.212:41419
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.253.254:34179
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988004637995-12279-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988004637995-12279-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988004637995-12279-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:47:25.008280 24673 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:47:25.008507 24673 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:47:25.008529 24673 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:47:25.009716 24673 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 03:47:25.009763 24673 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:47:25.009778 24673 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:47:25.009790 24673 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 03:47:25.009802 24673 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 03:47:25.011381 24673 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.253.212:41419
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988004637995-12279-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988004637995-12279-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.253.254:34179
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.254.keytab
--principal=kudu/127.11.253.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.253.254:34179
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988004637995-12279-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.253.254
--webserver_port=45713
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24673
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988004637995-12279-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:47:25.011667 24673 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:47:25.011921 24673 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:47:25.017072 24687 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:47:25.017084 24683 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:47:25.017094 24684 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:47:25.017688 24673 server_base.cc:1034] running on GCE node
I20260806 03:47:25.017987 24673 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:47:25.018247 24673 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:47:25.019416 24673 hybrid_clock.cc:648] HybridClock initialized: now 1785988045019403 us; error 30 us; skew 500 ppm
Aug 06 03:47:25 dist-test-slave-6ntk krb5kdc[23931](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988045, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.253.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:47:25.021984 24673 init.cc:377] Logged in from keytab as kudu/127.11.253.254@KRBTEST.COM (short username kudu)
I20260806 03:47:25.022428 24673 webserver.cc:460] Webserver started at http://127.11.253.254:45713/ using document root <none> and password file <none>
I20260806 03:47:25.022677 24673 fs_manager.cc:359] Metadata directory not provided
I20260806 03:47:25.022761 24673 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:47:25.024114 24673 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.001s
I20260806 03:47:25.024797 24701 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:47:25.024989 24673 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 03:47:25.025102 24673 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988004637995-12279-0/minicluster-data/master-0/data,/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988004637995-12279-0/minicluster-data/master-0/wal
uuid: "29a340ef16a049c09755c2af9a0098dc"
format_stamp: "Formatted at 2026-08-06 03:47:24 on dist-test-slave-6ntk"
server_key: "bba2519134dcce2eb4d20f52a19779c8"
server_key_iv: "7f019a3c5aab51c0fa88b2ae29a90c53"
server_key_version: "encryptionkey@0"
I20260806 03:47:25.025462 24673 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988004637995-12279-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988004637995-12279-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988004637995-12279-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:47:25.038800 24673 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:47:25.041195 24673 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:47:25.041409 24673 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:47:25.045794 24673 rpc_server.cc:307] RPC server started. Bound to: 127.11.253.254:34179
I20260806 03:47:25.045830 24772 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.253.254:34179 every 8 connection(s)
I20260806 03:47:25.046427 24673 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988004637995-12279-0/minicluster-data/master-0/data/info.pb
I20260806 03:47:25.047770 24773 sys_catalog.cc:263] Verifying existing consensus state
I20260806 03:47:25.048484 24773 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 29a340ef16a049c09755c2af9a0098dc: Bootstrap starting.
I20260806 03:47:25.049285 12279 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskToVWRe/build/release/bin/kudu as pid 24673
I20260806 03:47:25.049623 12279 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskToVWRe/build/release/bin/kudu
/tmp/dist-test-taskToVWRe/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988004637995-12279-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988004637995-12279-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988004637995-12279-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988004637995-12279-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.253.193:44747
--local_ip_for_outbound_sockets=127.11.253.193
--tserver_master_addrs=127.11.253.254:34179
--webserver_port=46095
--webserver_interface=127.11.253.193
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.193.keytab
--principal=kudu/127.11.253.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.253.212:41419
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.11.253.193
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988004637995-12279-0/krb5kdc/oryx_127.11.253.193.keytab with env {KRB5CCNAME=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988004637995-12279-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988004637995-12279-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988004637995-12279-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 03:47:25.060261 24773 log.cc:824] T 00000000000000000000000000000000 P 29a340ef16a049c09755c2af9a0098dc: Log is configured to *not* fsync() on all Append() calls
I20260806 03:47:25.062958 24773 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 29a340ef16a049c09755c2af9a0098dc: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 03:47:25.063380 24773 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 29a340ef16a049c09755c2af9a0098dc: Bootstrap complete.
I20260806 03:47:25.066879 24773 raft_consensus.cc:348] T 00000000000000000000000000000000 P 29a340ef16a049c09755c2af9a0098dc [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29a340ef16a049c09755c2af9a0098dc" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 34179 } }
I20260806 03:47:25.067210 24773 raft_consensus.cc:729] T 00000000000000000000000000000000 P 29a340ef16a049c09755c2af9a0098dc [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29a340ef16a049c09755c2af9a0098dc, State: Initialized, Role: FOLLOWER
I20260806 03:47:25.067373 24773 consensus_queue.cc:260] T 00000000000000000000000000000000 P 29a340ef16a049c09755c2af9a0098dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29a340ef16a049c09755c2af9a0098dc" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 34179 } }
I20260806 03:47:25.067437 24773 raft_consensus.cc:388] T 00000000000000000000000000000000 P 29a340ef16a049c09755c2af9a0098dc [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 03:47:25.067474 24773 raft_consensus.cc:482] T 00000000000000000000000000000000 P 29a340ef16a049c09755c2af9a0098dc [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 03:47:25.067517 24773 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 29a340ef16a049c09755c2af9a0098dc [term 1 FOLLOWER]: Advancing to term 2
I20260806 03:47:25.068955 24773 raft_consensus.cc:504] T 00000000000000000000000000000000 P 29a340ef16a049c09755c2af9a0098dc [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29a340ef16a049c09755c2af9a0098dc" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 34179 } }
I20260806 03:47:25.069068 24773 leader_election.cc:302] T 00000000000000000000000000000000 P 29a340ef16a049c09755c2af9a0098dc [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 29a340ef16a049c09755c2af9a0098dc; no voters:
I20260806 03:47:25.069242 24773 leader_election.cc:288] T 00000000000000000000000000000000 P 29a340ef16a049c09755c2af9a0098dc [CANDIDATE]: Term 2 election: Requested vote from peers
I20260806 03:47:25.069309 24778 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 29a340ef16a049c09755c2af9a0098dc [term 2 FOLLOWER]: Leader election won for term 2
I20260806 03:47:25.069487 24773 sys_catalog.cc:564] T 00000000000000000000000000000000 P 29a340ef16a049c09755c2af9a0098dc [sys.catalog]: configured and running, proceeding with master startup.
I20260806 03:47:25.069540 24778 raft_consensus.cc:686] T 00000000000000000000000000000000 P 29a340ef16a049c09755c2af9a0098dc [term 2 LEADER]: Becoming Leader. State: Replica: 29a340ef16a049c09755c2af9a0098dc, State: Running, Role: LEADER
I20260806 03:47:25.069638 24778 consensus_queue.cc:237] T 00000000000000000000000000000000 P 29a340ef16a049c09755c2af9a0098dc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29a340ef16a049c09755c2af9a0098dc" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 34179 } }
I20260806 03:47:25.069943 24779 sys_catalog.cc:455] T 00000000000000000000000000000000 P 29a340ef16a049c09755c2af9a0098dc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "29a340ef16a049c09755c2af9a0098dc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29a340ef16a049c09755c2af9a0098dc" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 34179 } } }
I20260806 03:47:25.070027 24779 sys_catalog.cc:458] T 00000000000000000000000000000000 P 29a340ef16a049c09755c2af9a0098dc [sys.catalog]: This master's current role is: LEADER
I20260806 03:47:25.070283 24781 sys_catalog.cc:455] T 00000000000000000000000000000000 P 29a340ef16a049c09755c2af9a0098dc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 29a340ef16a049c09755c2af9a0098dc. Latest consensus state: current_term: 2 leader_uuid: "29a340ef16a049c09755c2af9a0098dc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29a340ef16a049c09755c2af9a0098dc" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 34179 } } }
I20260806 03:47:25.070361 24781 sys_catalog.cc:458] T 00000000000000000000000000000000 P 29a340ef16a049c09755c2af9a0098dc [sys.catalog]: This master's current role is: LEADER
I20260806 03:47:25.070477 24790 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 03:47:25.071002 24790 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 03:47:25.071141 24790 catalog_manager.cc:1259] Loaded cluster ID: 4ca64d742e8e46c8be030a36bcb6fa62
I20260806 03:47:25.071175 24790 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 03:47:25.072486 24790 catalog_manager.cc:1504] Loading token signing keys...
I20260806 03:47:25.072659 24790 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 29a340ef16a049c09755c2af9a0098dc: Loaded TSK: 0
I20260806 03:47:25.072882 24790 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 03:47:25.143016 24776 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:47:25.143270 24776 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:47:25.143309 24776 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:47:25.144490 24776 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:47:25.144562 24776 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:47:25.144606 24776 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.253.193
I20260806 03:47:25.146136 24776 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.253.212:41419
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988004637995-12279-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988004637995-12279-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988004637995-12279-0/krb5kdc/oryx_127.11.253.193.keytab
--principal=oryx/127.11.253.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.253.193:44747
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988004637995-12279-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.253.193
--webserver_port=46095
--tserver_master_addrs=127.11.253.254:34179
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24776
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.253.193
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988004637995-12279-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:47:25.146426 24776 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:47:25.146667 24776 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:47:25.152127 24815 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:47:25.152104 24820 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:47:25.152321 24776 server_base.cc:1034] running on GCE node
W20260806 03:47:25.152386 24816 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:47:25.152647 24776 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:47:25.152916 24776 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:47:25.154088 24776 hybrid_clock.cc:648] HybridClock initialized: now 1785988045154071 us; error 35 us; skew 500 ppm
Aug 06 03:47:25 dist-test-slave-6ntk krb5kdc[23931](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988045, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.253.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:47:25.157085 24776 init.cc:377] Logged in from keytab as oryx/127.11.253.193@KRBTEST.COM (short username oryx)
I20260806 03:47:25.157652 24776 webserver.cc:460] Webserver started at http://127.11.253.193:46095/ using document root <none> and password file <none>
I20260806 03:47:25.157981 24776 fs_manager.cc:359] Metadata directory not provided
I20260806 03:47:25.158072 24776 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:47:25.159917 24776 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 03:47:25.160645 24832 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:47:25.160868 24776 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 03:47:25.160974 24776 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988004637995-12279-0/minicluster-data/ts-0/data,/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988004637995-12279-0/minicluster-data/ts-0/wal
uuid: "7c2c120f6cd048798f67ee612082b30e"
format_stamp: "Formatted at 2026-08-06 03:47:24 on dist-test-slave-6ntk"
server_key: "53edd8b0540505de18d90992e8348bd1"
server_key_iv: "4754f07e99c8fca2327971f96f012773"
server_key_version: "encryptionkey@0"
I20260806 03:47:25.161403 24776 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988004637995-12279-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988004637995-12279-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988004637995-12279-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:47:25.182519 24776 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:47:25.185072 24776 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:47:25.185263 24776 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:47:25.185539 24776 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 03:47:25.185989 24776 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 03:47:25.186050 24776 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 03:47:25.186084 24776 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 03:47:25.186101 24776 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 03:47:25.193603 24776 rpc_server.cc:307] RPC server started. Bound to: 127.11.253.193:44747
I20260806 03:47:25.193639 24994 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.253.193:44747 every 8 connection(s)
I20260806 03:47:25.194113 24776 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988004637995-12279-0/minicluster-data/ts-0/data/info.pb
I20260806 03:47:25.196981 12279 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskToVWRe/build/release/bin/kudu as pid 24776
I20260806 03:47:25.197341 12279 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskToVWRe/build/release/bin/kudu
/tmp/dist-test-taskToVWRe/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988004637995-12279-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988004637995-12279-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988004637995-12279-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988004637995-12279-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.253.194:41175
--local_ip_for_outbound_sockets=127.11.253.194
--tserver_master_addrs=127.11.253.254:34179
--webserver_port=35353
--webserver_interface=127.11.253.194
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.194.keytab
--principal=kudu/127.11.253.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.253.212:41419
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.11.253.194
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988004637995-12279-0/krb5kdc/oryx_127.11.253.194.keytab with env {KRB5CCNAME=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988004637995-12279-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988004637995-12279-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988004637995-12279-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 03:47:25 dist-test-slave-6ntk krb5kdc[23931](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.253.193@KRBTEST.COM for oryx/127.11.253.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 03:47:25 dist-test-slave-6ntk krb5kdc[23931](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.253.193@KRBTEST.COM for oryx/127.11.253.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 03:47:25.205123 24997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:25.194896 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:34179 (local address 127.11.253.193:42993)
0806 03:47:25.195093 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0806 03:47:25.195145 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0806 03:47:25.195658 (+ 513us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:47:25.196124 (+ 466us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:47:25.196127 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:47:25.196743 (+ 616us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:47:25.197046 (+ 303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:25.197054 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:25.198805 (+ 1751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:25.198806 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:25.200064 (+ 1258us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:25.200067 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:25.200958 (+ 891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:25.200959 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:25.201104 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:25.201759 (+ 655us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:47:25.201769 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:47:25.204878 (+ 3109us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.253.254:34179: Server oryx/127.11.253.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":90,"threads_started":1}
W20260806 03:47:25.205394 24995 heartbeater.cc:644] Failed to heartbeat to 127.11.253.254:34179 (0 consecutive failures): Not authorized: Failed to ping master at 127.11.253.254:34179: Client connection negotiation failed: client connection to 127.11.253.254:34179: Server oryx/127.11.253.254@KRBTEST.COM not found in Kerberos database
W20260806 03:47:25.206117 25000 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 03:47:25.194946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.253.193:42993 (local address 127.11.253.254:34179)
0806 03:47:25.195178 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0806 03:47:25.195187 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 03:47:25.195230 (+ 43us) server_negotiation.cc:407] Connection header received
0806 03:47:25.195898 (+ 668us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:47:25.195904 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:47:25.195947 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:47:25.196085 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:47:25.197201 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:25.198683 (+ 1482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:25.200180 (+ 1497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:25.200837 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:25.200953 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:25.205052 (+ 4099us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.253.193:42993: BlockingRecv error: recv got EOF from 127.11.253.193:42993 (error 108)
Metrics: {"server-negotiator.queue_time_us":185,"thread_start_us":109,"threads_started":1}
Aug 06 03:47:25 dist-test-slave-6ntk krb5kdc[23931](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.253.193@KRBTEST.COM for oryx/127.11.253.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 03:47:25 dist-test-slave-6ntk krb5kdc[23931](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.253.193@KRBTEST.COM for oryx/127.11.253.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 03:47:25.211985 24997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:25.205663 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:34179 (local address 127.11.253.193:51279)
0806 03:47:25.205910 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0806 03:47:25.205922 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 03:47:25.205992 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:47:25.206419 (+ 427us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:47:25.206421 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:47:25.206579 (+ 158us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:47:25.206639 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:25.206641 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:25.207648 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:25.207649 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:25.208479 (+ 830us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:25.208481 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:25.209029 (+ 548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:25.209030 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:25.209097 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:25.209651 (+ 554us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:47:25.209656 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:47:25.211904 (+ 2248us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.253.254:34179: Server oryx/127.11.253.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":185}
W20260806 03:47:25.212100 25000 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 03:47:25.205727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.253.193:51279 (local address 127.11.253.254:34179)
0806 03:47:25.206290 (+ 563us) server_negotiation.cc:206] Beginning negotiation
0806 03:47:25.206292 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:47:25.206302 (+ 10us) server_negotiation.cc:407] Connection header received
0806 03:47:25.206328 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:47:25.206329 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:47:25.206345 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:47:25.206412 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:47:25.206740 (+ 328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:25.207544 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:25.208582 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:25.208928 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:25.209009 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:25.212034 (+ 3025us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.253.193:51279: BlockingRecv error: recv got EOF from 127.11.253.193:51279 (error 108)
Metrics: {"server-negotiator.queue_time_us":542,"thread_start_us":100,"threads_started":1}
Aug 06 03:47:25 dist-test-slave-6ntk krb5kdc[23931](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.253.193@KRBTEST.COM for oryx/127.11.253.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 03:47:25 dist-test-slave-6ntk krb5kdc[23931](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.253.193@KRBTEST.COM for oryx/127.11.253.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 03:47:25.217933 24997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:25.212742 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:34179 (local address 127.11.253.193:47155)
0806 03:47:25.212792 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0806 03:47:25.212803 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 03:47:25.212837 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:47:25.212990 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:47:25.212991 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:47:25.213155 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:47:25.213193 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:25.213195 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:25.214058 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:25.214059 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:25.214755 (+ 696us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:25.214758 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:25.215275 (+ 517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:25.215276 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:25.215328 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:25.215828 (+ 500us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:47:25.215832 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:47:25.217836 (+ 2004us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.253.254:34179: Server oryx/127.11.253.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":34}
W20260806 03:47:25.218037 25000 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 03:47:25.212821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.253.193:47155 (local address 127.11.253.254:34179)
0806 03:47:25.212858 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0806 03:47:25.212860 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:47:25.212868 (+ 8us) server_negotiation.cc:407] Connection header received
0806 03:47:25.212894 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:47:25.212895 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:47:25.212905 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:47:25.212967 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:47:25.213280 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:25.213969 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:25.214855 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:25.215173 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:25.215246 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:25.217956 (+ 2710us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.253.193:47155: BlockingRecv error: recv got EOF from 127.11.253.193:47155 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260806 03:47:25.218078 24995 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 03:47:25.288524 25001 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:47:25.288765 25001 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:47:25.288791 25001 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:47:25.290086 25001 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:47:25.290138 25001 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:47:25.290166 25001 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.253.194
I20260806 03:47:25.291839 25001 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.253.212:41419
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988004637995-12279-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988004637995-12279-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988004637995-12279-0/krb5kdc/oryx_127.11.253.194.keytab
--principal=oryx/127.11.253.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.253.194:41175
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988004637995-12279-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.253.194
--webserver_port=35353
--tserver_master_addrs=127.11.253.254:34179
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25001
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.253.194
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988004637995-12279-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:47:25.292122 25001 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:47:25.292414 25001 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:47:25.297778 25009 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:47:25.297773 25010 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:47:25.297770 25013 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:47:25.297855 25001 server_base.cc:1034] running on GCE node
I20260806 03:47:25.298245 25001 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:47:25.298502 25001 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:47:25.299676 25001 hybrid_clock.cc:648] HybridClock initialized: now 1785988045299664 us; error 26 us; skew 500 ppm
Aug 06 03:47:25 dist-test-slave-6ntk krb5kdc[23931](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988045, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.253.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:47:25.302311 25001 init.cc:377] Logged in from keytab as oryx/127.11.253.194@KRBTEST.COM (short username oryx)
I20260806 03:47:25.302791 25001 webserver.cc:460] Webserver started at http://127.11.253.194:35353/ using document root <none> and password file <none>
I20260806 03:47:25.303041 25001 fs_manager.cc:359] Metadata directory not provided
I20260806 03:47:25.303131 25001 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:47:25.304517 25001 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 03:47:25.305181 25023 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:47:25.305424 25001 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 03:47:25.305533 25001 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988004637995-12279-0/minicluster-data/ts-1/data,/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988004637995-12279-0/minicluster-data/ts-1/wal
uuid: "6aeb2de5b6864b51abf7d0064007dbec"
format_stamp: "Formatted at 2026-08-06 03:47:24 on dist-test-slave-6ntk"
server_key: "99f272b74fe9cbea9c9d9c9ee58288a5"
server_key_iv: "39dc10b3036eb141e9f79906ae3a3b2a"
server_key_version: "encryptionkey@0"
I20260806 03:47:25.305914 25001 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988004637995-12279-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988004637995-12279-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988004637995-12279-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:47:25.325502 25001 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:47:25.328024 25001 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:47:25.328207 25001 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:47:25.328477 25001 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 03:47:25.328815 25001 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 03:47:25.328879 25001 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 03:47:25.328915 25001 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 03:47:25.328944 25001 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 03:47:25.335109 25001 rpc_server.cc:307] RPC server started. Bound to: 127.11.253.194:41175
I20260806 03:47:25.335160 25204 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.253.194:41175 every 8 connection(s)
I20260806 03:47:25.335518 25001 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988004637995-12279-0/minicluster-data/ts-1/data/info.pb
Aug 06 03:47:25 dist-test-slave-6ntk krb5kdc[23931](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.253.194@KRBTEST.COM for oryx/127.11.253.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 03:47:25 dist-test-slave-6ntk krb5kdc[23931](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.253.194@KRBTEST.COM for oryx/127.11.253.254@KRBTEST.COM, Server not found in Kerberos database
W20260806 03:47:25.343417 25000 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 03:47:25.336298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.253.194:47793 (local address 127.11.253.254:34179)
0806 03:47:25.336375 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0806 03:47:25.336378 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 03:47:25.336602 (+ 224us) server_negotiation.cc:407] Connection header received
0806 03:47:25.337168 (+ 566us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:47:25.337169 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:47:25.337187 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:47:25.337239 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:47:25.338308 (+ 1069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:25.338811 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:25.339847 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:25.340050 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:25.340135 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:25.343357 (+ 3222us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.253.194:47793: BlockingRecv error: recv got EOF from 127.11.253.194:47793 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
I20260806 03:47:25.343487 25208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:25.336306 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:34179 (local address 127.11.253.194:47793)
0806 03:47:25.336477 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0806 03:47:25.336514 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0806 03:47:25.337043 (+ 529us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:47:25.337314 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:47:25.337318 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:47:25.338003 (+ 685us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:47:25.338201 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:25.338207 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:25.338921 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:25.338922 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:25.339733 (+ 811us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:25.339735 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:25.340143 (+ 408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:25.340144 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:25.340274 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:25.340881 (+ 607us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:47:25.340890 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:47:25.343265 (+ 2375us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.253.254:34179: Server oryx/127.11.253.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":68,"threads_started":1}
W20260806 03:47:25.343879 25206 heartbeater.cc:644] Failed to heartbeat to 127.11.253.254:34179 (0 consecutive failures): Not authorized: Failed to ping master at 127.11.253.254:34179: Client connection negotiation failed: client connection to 127.11.253.254:34179: Server oryx/127.11.253.254@KRBTEST.COM not found in Kerberos database
I20260806 03:47:25.344619 12279 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskToVWRe/build/release/bin/kudu as pid 25001
I20260806 03:47:25.344990 12279 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskToVWRe/build/release/bin/kudu
/tmp/dist-test-taskToVWRe/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988004637995-12279-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988004637995-12279-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988004637995-12279-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988004637995-12279-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.253.195:40465
--local_ip_for_outbound_sockets=127.11.253.195
--tserver_master_addrs=127.11.253.254:34179
--webserver_port=33237
--webserver_interface=127.11.253.195
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.195.keytab
--principal=kudu/127.11.253.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.253.212:41419
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.11.253.195
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988004637995-12279-0/krb5kdc/oryx_127.11.253.195.keytab with env {KRB5CCNAME=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988004637995-12279-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988004637995-12279-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988004637995-12279-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 03:47:25 dist-test-slave-6ntk krb5kdc[23931](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.253.194@KRBTEST.COM for oryx/127.11.253.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 03:47:25 dist-test-slave-6ntk krb5kdc[23931](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.253.194@KRBTEST.COM for oryx/127.11.253.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 03:47:25.349504 25208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:25.344201 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:34179 (local address 127.11.253.194:58101)
0806 03:47:25.344306 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0806 03:47:25.344319 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 03:47:25.344401 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:47:25.344573 (+ 172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:47:25.344574 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:47:25.344715 (+ 141us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:47:25.344775 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:25.344777 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:25.345612 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:25.345613 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:25.346171 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:25.346173 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:25.346649 (+ 476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:25.346650 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:25.346722 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:25.347204 (+ 482us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:47:25.347210 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:47:25.349455 (+ 2245us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.253.254:34179: Server oryx/127.11.253.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":52}
W20260806 03:47:25.349730 25000 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 03:47:25.344336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.253.194:58101 (local address 127.11.253.254:34179)
0806 03:47:25.344420 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0806 03:47:25.344422 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:47:25.344434 (+ 12us) server_negotiation.cc:407] Connection header received
0806 03:47:25.344466 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:47:25.344467 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:47:25.344481 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:47:25.344554 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:47:25.344854 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:25.345523 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:25.346253 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:25.346567 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:25.346653 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:25.349643 (+ 2990us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.253.194:58101: BlockingRecv error: recv got EOF from 127.11.253.194:58101 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Aug 06 03:47:25 dist-test-slave-6ntk krb5kdc[23931](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.253.194@KRBTEST.COM for oryx/127.11.253.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 03:47:25 dist-test-slave-6ntk krb5kdc[23931](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.253.194@KRBTEST.COM for oryx/127.11.253.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 03:47:25.354584 25208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:25.349755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:34179 (local address 127.11.253.194:52233)
0806 03:47:25.349911 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0806 03:47:25.349921 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 03:47:25.349976 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:47:25.350149 (+ 173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:47:25.350150 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:47:25.350262 (+ 112us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:47:25.350289 (+ 27us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:25.350290 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:25.351288 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:25.351289 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:25.351791 (+ 502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:25.351793 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:25.352331 (+ 538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:25.352331 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:25.352369 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:25.352822 (+ 453us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:47:25.352827 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:47:25.354485 (+ 1658us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.253.254:34179: Server oryx/127.11.253.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":80,"threads_started":1}
W20260806 03:47:25.354700 25002 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 03:47:25.349822 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.253.194:52233 (local address 127.11.253.254:34179)
0806 03:47:25.349924 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0806 03:47:25.349926 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:47:25.349979 (+ 53us) server_negotiation.cc:407] Connection header received
0806 03:47:25.350030 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:47:25.350032 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:47:25.350046 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:47:25.350128 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:47:25.350374 (+ 246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:25.351183 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:25.351884 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:25.352229 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:25.352308 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:25.354613 (+ 2305us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.253.194:52233: BlockingRecv error: recv got EOF from 127.11.253.194:52233 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
W20260806 03:47:25.354722 25206 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 03:47:25.431787 25209 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:47:25.432001 25209 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:47:25.432026 25209 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:47:25.433182 25209 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:47:25.433218 25209 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:47:25.433244 25209 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.253.195
I20260806 03:47:25.434587 25209 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.253.212:41419
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988004637995-12279-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988004637995-12279-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988004637995-12279-0/krb5kdc/oryx_127.11.253.195.keytab
--principal=oryx/127.11.253.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.253.195:40465
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988004637995-12279-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.253.195
--webserver_port=33237
--tserver_master_addrs=127.11.253.254:34179
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25209
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.253.195
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988004637995-12279-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:47:25.434767 25209 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:47:25.434947 25209 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:47:25.440176 25224 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:47:25.440183 25217 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:47:25.440186 25218 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:47:25.440557 25209 server_base.cc:1034] running on GCE node
I20260806 03:47:25.440793 25209 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:47:25.441054 25209 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:47:25.442202 25209 hybrid_clock.cc:648] HybridClock initialized: now 1785988045442173 us; error 50 us; skew 500 ppm
Aug 06 03:47:25 dist-test-slave-6ntk krb5kdc[23931](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988045, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.253.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:47:25.444991 25209 init.cc:377] Logged in from keytab as oryx/127.11.253.195@KRBTEST.COM (short username oryx)
I20260806 03:47:25.445447 25209 webserver.cc:460] Webserver started at http://127.11.253.195:33237/ using document root <none> and password file <none>
I20260806 03:47:25.445691 25209 fs_manager.cc:359] Metadata directory not provided
I20260806 03:47:25.445811 25209 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:47:25.447242 25209 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 03:47:25.447901 25233 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:47:25.448089 25209 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 03:47:25.448203 25209 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988004637995-12279-0/minicluster-data/ts-2/data,/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988004637995-12279-0/minicluster-data/ts-2/wal
uuid: "a1b25c1ae9b24bf894980f3e84b6641d"
format_stamp: "Formatted at 2026-08-06 03:47:24 on dist-test-slave-6ntk"
server_key: "33a5f843dab1cccdfec8ff5713387529"
server_key_iv: "423675b2cfe9f88b332f37a977ec7170"
server_key_version: "encryptionkey@0"
I20260806 03:47:25.448553 25209 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988004637995-12279-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988004637995-12279-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988004637995-12279-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:47:25.460475 25209 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:47:25.462980 25209 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:47:25.463162 25209 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:47:25.463420 25209 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 03:47:25.463742 25209 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 03:47:25.463811 25209 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 03:47:25.463857 25209 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 03:47:25.463886 25209 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 03:47:25.470045 25209 rpc_server.cc:307] RPC server started. Bound to: 127.11.253.195:40465
I20260806 03:47:25.470153 25411 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.253.195:40465 every 8 connection(s)
I20260806 03:47:25.470518 25209 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988004637995-12279-0/minicluster-data/ts-2/data/info.pb
I20260806 03:47:25.471794 12279 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskToVWRe/build/release/bin/kudu as pid 25209
Aug 06 03:47:25 dist-test-slave-6ntk krb5kdc[23931](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.253.195@KRBTEST.COM for oryx/127.11.253.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 03:47:25 dist-test-slave-6ntk krb5kdc[23931](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.253.195@KRBTEST.COM for oryx/127.11.253.254@KRBTEST.COM, Server not found in Kerberos database
W20260806 03:47:25.480422 25002 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 03:47:25.471274 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.253.195:40537 (local address 127.11.253.254:34179)
0806 03:47:25.471355 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0806 03:47:25.471357 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:47:25.471571 (+ 214us) server_negotiation.cc:407] Connection header received
0806 03:47:25.472149 (+ 578us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:47:25.472150 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:47:25.472164 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:47:25.472354 (+ 190us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:47:25.473734 (+ 1380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:25.474439 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:25.475923 (+ 1484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:25.476101 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:25.476153 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:25.480363 (+ 4210us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.253.195:40537: BlockingRecv error: recv got EOF from 127.11.253.195:40537 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
I20260806 03:47:25.480499 25415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:25.471280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:34179 (local address 127.11.253.195:40537)
0806 03:47:25.471459 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0806 03:47:25.471495 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0806 03:47:25.472022 (+ 527us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:47:25.472493 (+ 471us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:47:25.472498 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:47:25.473366 (+ 868us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:47:25.473639 (+ 273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:25.473645 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:25.474569 (+ 924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:25.474570 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:25.475836 (+ 1266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:25.475841 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:25.476194 (+ 353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:25.476195 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:25.476357 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:25.476892 (+ 535us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:47:25.476905 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:47:25.480286 (+ 3381us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.253.254:34179: Server oryx/127.11.253.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":109,"threads_started":1}
I20260806 03:47:25.480763 25000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:25.472528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33654 (local address 127.11.253.254:34179)
0806 03:47:25.472589 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0806 03:47:25.472591 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:47:25.472599 (+ 8us) server_negotiation.cc:407] Connection header received
0806 03:47:25.472619 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:47:25.472620 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:47:25.472628 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:47:25.472685 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:47:25.473524 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:25.474278 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:25.475524 (+ 1246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:25.475703 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:25.475755 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:25.477127 (+ 1372us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:47:25.477132 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:47:25.477132 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:47:25.477145 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:47:25.478894 (+ 1749us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:47:25.479636 (+ 742us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:47:25.479639 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:47:25.479640 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:47:25.479734 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:47:25.480003 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:47:25.480004 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:47:25.480004 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:47:25.480254 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:47:25.480417 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0806 03:47:25.480533 (+ 116us) server_negotiation.cc:299] Negotiation successful
0806 03:47:25.480650 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
W20260806 03:47:25.480849 25412 heartbeater.cc:644] Failed to heartbeat to 127.11.253.254:34179 (0 consecutive failures): Not authorized: Failed to ping master at 127.11.253.254:34179: Client connection negotiation failed: client connection to 127.11.253.254:34179: Server oryx/127.11.253.254@KRBTEST.COM not found in Kerberos database
Aug 06 03:47:25 dist-test-slave-6ntk krb5kdc[23931](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.253.195@KRBTEST.COM for oryx/127.11.253.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 03:47:25 dist-test-slave-6ntk krb5kdc[23931](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.253.195@KRBTEST.COM for oryx/127.11.253.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 03:47:25.485337 25415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:25.481086 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:34179 (local address 127.11.253.195:55467)
0806 03:47:25.481131 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0806 03:47:25.481140 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 03:47:25.481177 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:47:25.481297 (+ 120us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:47:25.481298 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:47:25.481484 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:47:25.481537 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:25.481539 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:25.482332 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:25.482333 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:25.482830 (+ 497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:25.482832 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:25.483173 (+ 341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:25.483173 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:25.483223 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:25.483673 (+ 450us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:47:25.483677 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:47:25.485290 (+ 1613us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.253.254:34179: Server oryx/127.11.253.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":32}
W20260806 03:47:25.485448 25000 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 03:47:25.481146 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.253.195:55467 (local address 127.11.253.254:34179)
0806 03:47:25.481204 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0806 03:47:25.481206 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:47:25.481216 (+ 10us) server_negotiation.cc:407] Connection header received
0806 03:47:25.481238 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:47:25.481239 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:47:25.481246 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:47:25.481292 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:47:25.481640 (+ 348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:25.482228 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:25.482914 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:25.483099 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:25.483154 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:25.485381 (+ 2227us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.253.195:55467: BlockingRecv error: recv got EOF from 127.11.253.195:55467 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
Aug 06 03:47:25 dist-test-slave-6ntk krb5kdc[23931](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.253.195@KRBTEST.COM for oryx/127.11.253.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 03:47:25 dist-test-slave-6ntk krb5kdc[23931](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.253.195@KRBTEST.COM for oryx/127.11.253.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 03:47:25.489825 25415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:25.485673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:34179 (local address 127.11.253.195:33045)
0806 03:47:25.485729 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0806 03:47:25.485739 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 03:47:25.485803 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:47:25.485985 (+ 182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:47:25.485986 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:47:25.486111 (+ 125us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:47:25.486135 (+ 24us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:25.486137 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:25.486759 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:25.486760 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:25.487255 (+ 495us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:25.487257 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:25.487619 (+ 362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:25.487620 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:25.487671 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:25.488070 (+ 399us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:47:25.488075 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:47:25.489727 (+ 1652us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.253.254:34179: Server oryx/127.11.253.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":39}
W20260806 03:47:25.489933 25000 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 03:47:25.485760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.253.195:33045 (local address 127.11.253.254:34179)
0806 03:47:25.485828 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0806 03:47:25.485830 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:47:25.485854 (+ 24us) server_negotiation.cc:407] Connection header received
0806 03:47:25.485889 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:47:25.485889 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:47:25.485899 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:47:25.485982 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:47:25.486219 (+ 237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:25.486676 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:25.487348 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:25.487543 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:25.487622 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:25.489812 (+ 2190us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.253.195:33045: BlockingRecv error: recv got EOF from 127.11.253.195:33045 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
W20260806 03:47:25.490051 25412 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 06 03:47:26 dist-test-slave-6ntk krb5kdc[23931](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.253.193@KRBTEST.COM for oryx/127.11.253.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 03:47:26 dist-test-slave-6ntk krb5kdc[23931](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.253.193@KRBTEST.COM for oryx/127.11.253.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 03:47:26.223702 25420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:26.218507 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:34179 (local address 127.11.253.193:34657)
0806 03:47:26.218678 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0806 03:47:26.218692 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 03:47:26.218759 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:47:26.218898 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:47:26.218899 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:47:26.219070 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:47:26.219122 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:26.219124 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:26.219858 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:26.219858 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:26.220419 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:26.220421 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:26.220850 (+ 429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:26.220851 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:26.220895 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:26.221563 (+ 668us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:47:26.221568 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:47:26.223632 (+ 2064us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.253.254:34179: Server oryx/127.11.253.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":94,"threads_started":1}
W20260806 03:47:26.223920 25421 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 03:47:26.218570 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.253.193:34657 (local address 127.11.253.254:34179)
0806 03:47:26.218759 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0806 03:47:26.218761 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:47:26.218771 (+ 10us) server_negotiation.cc:407] Connection header received
0806 03:47:26.218800 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:47:26.218800 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:47:26.218813 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:47:26.218884 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:47:26.219198 (+ 314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:26.219770 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:26.220510 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:26.220757 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:26.220829 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:26.223820 (+ 2991us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.253.193:34657: BlockingRecv error: recv got EOF from 127.11.253.193:34657 (error 108)
Metrics: {"server-negotiator.queue_time_us":165,"thread_start_us":95,"threads_started":1}
Aug 06 03:47:26 dist-test-slave-6ntk krb5kdc[23931](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.253.194@KRBTEST.COM for oryx/127.11.253.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 03:47:26 dist-test-slave-6ntk krb5kdc[23931](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.253.194@KRBTEST.COM for oryx/127.11.253.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 03:47:26.359946 25423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:26.355140 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:34179 (local address 127.11.253.194:60011)
0806 03:47:26.355304 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0806 03:47:26.355331 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0806 03:47:26.355403 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:47:26.355533 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:47:26.355534 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:47:26.355705 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:47:26.355758 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:26.355760 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:26.356384 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:26.356384 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:26.356925 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:26.356927 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:26.357250 (+ 323us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:26.357251 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:26.357292 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:26.357776 (+ 484us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:47:26.357780 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:47:26.359872 (+ 2092us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.253.254:34179: Server oryx/127.11.253.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":78,"threads_started":1}
W20260806 03:47:26.360087 25421 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 03:47:26.355263 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.253.194:60011 (local address 127.11.253.254:34179)
0806 03:47:26.355333 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0806 03:47:26.355335 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:47:26.355400 (+ 65us) server_negotiation.cc:407] Connection header received
0806 03:47:26.355448 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:47:26.355449 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:47:26.355459 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:47:26.355510 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:47:26.355818 (+ 308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:26.356308 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:26.356996 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:26.357180 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:26.357231 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:26.359974 (+ 2743us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.253.194:60011: BlockingRecv error: recv got EOF from 127.11.253.194:60011 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
Aug 06 03:47:26 dist-test-slave-6ntk krb5kdc[23931](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.253.195@KRBTEST.COM for oryx/127.11.253.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 03:47:26 dist-test-slave-6ntk krb5kdc[23931](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.253.195@KRBTEST.COM for oryx/127.11.253.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 03:47:26.495587 25424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:26.490570 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:34179 (local address 127.11.253.195:41773)
0806 03:47:26.490746 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0806 03:47:26.490756 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 03:47:26.490838 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:47:26.491010 (+ 172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:47:26.491011 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:47:26.491200 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:47:26.491248 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:26.491250 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:26.491941 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:26.491942 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:26.492501 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:26.492503 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:26.492930 (+ 427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:26.492931 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:26.492997 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:26.493557 (+ 560us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:47:26.493567 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:47:26.495473 (+ 1906us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.253.254:34179: Server oryx/127.11.253.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":93,"threads_started":1}
W20260806 03:47:26.495623 25421 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 03:47:26.490620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.253.195:41773 (local address 127.11.253.254:34179)
0806 03:47:26.490710 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0806 03:47:26.490714 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 03:47:26.490823 (+ 109us) server_negotiation.cc:407] Connection header received
0806 03:47:26.490896 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:47:26.490897 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:47:26.490913 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:47:26.490999 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:47:26.491338 (+ 339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:26.491834 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:26.492600 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:26.492801 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:26.492930 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:26.495558 (+ 2628us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.253.195:41773: BlockingRecv error: recv got EOF from 127.11.253.195:41773 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Aug 06 03:47:27 dist-test-slave-6ntk krb5kdc[23931](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.253.193@KRBTEST.COM for oryx/127.11.253.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 03:47:27 dist-test-slave-6ntk krb5kdc[23931](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.253.193@KRBTEST.COM for oryx/127.11.253.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 03:47:27.229120 25426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:27.224278 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:34179 (local address 127.11.253.193:52307)
0806 03:47:27.224450 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0806 03:47:27.224467 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 03:47:27.224539 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:47:27.224712 (+ 173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:47:27.224713 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:47:27.224915 (+ 202us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:47:27.224968 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:27.224970 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:27.225651 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:27.225651 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:27.226256 (+ 605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:27.226258 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:27.226618 (+ 360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:27.226619 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:27.226659 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:27.227131 (+ 472us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:47:27.227135 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:47:27.229060 (+ 1925us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.253.254:34179: Server oryx/127.11.253.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":120,"threads_started":1}
W20260806 03:47:27.229269 25427 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 03:47:27.224337 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.253.193:52307 (local address 127.11.253.254:34179)
0806 03:47:27.224495 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0806 03:47:27.224498 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 03:47:27.224509 (+ 11us) server_negotiation.cc:407] Connection header received
0806 03:47:27.224603 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:47:27.224603 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:47:27.224618 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:47:27.224690 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:47:27.225050 (+ 360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:27.225581 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:27.226360 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:27.226548 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:27.226602 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:27.229175 (+ 2573us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.253.193:52307: BlockingRecv error: recv got EOF from 127.11.253.193:52307 (error 108)
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":104,"threads_started":1}
Aug 06 03:47:27 dist-test-slave-6ntk krb5kdc[23931](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.253.194@KRBTEST.COM for oryx/127.11.253.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 03:47:27 dist-test-slave-6ntk krb5kdc[23931](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.253.194@KRBTEST.COM for oryx/127.11.253.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 03:47:27.365420 25428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:27.360540 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:34179 (local address 127.11.253.194:41727)
0806 03:47:27.360700 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0806 03:47:27.360716 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 03:47:27.360774 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:47:27.360938 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:47:27.360939 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:47:27.361138 (+ 199us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:47:27.361188 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:27.361189 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:27.361968 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:27.361969 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:27.362482 (+ 513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:27.362483 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:27.362814 (+ 331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:27.362815 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:27.362860 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:27.363321 (+ 461us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:47:27.363325 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:47:27.365357 (+ 2032us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.253.254:34179: Server oryx/127.11.253.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":100,"threads_started":1}
W20260806 03:47:27.365489 25427 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 03:47:27.360634 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.253.194:41727 (local address 127.11.253.254:34179)
0806 03:47:27.360694 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0806 03:47:27.360696 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:47:27.360761 (+ 65us) server_negotiation.cc:407] Connection header received
0806 03:47:27.360838 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:47:27.360839 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:47:27.360850 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:47:27.360902 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:47:27.361273 (+ 371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:27.361845 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:27.362559 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:27.362745 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:27.362799 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:27.365439 (+ 2640us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.253.194:41727: BlockingRecv error: recv got EOF from 127.11.253.194:41727 (error 108)
Metrics: {"server-negotiator.queue_time_us":26}
Aug 06 03:47:27 dist-test-slave-6ntk krb5kdc[23931](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.253.195@KRBTEST.COM for oryx/127.11.253.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 03:47:27 dist-test-slave-6ntk krb5kdc[23931](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.253.195@KRBTEST.COM for oryx/127.11.253.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 03:47:27.501175 25429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:27.496170 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:34179 (local address 127.11.253.195:33009)
0806 03:47:27.496342 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0806 03:47:27.496355 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 03:47:27.496426 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:47:27.496601 (+ 175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:47:27.496602 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:47:27.496878 (+ 276us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:47:27.496941 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:27.496943 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:27.497664 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:27.497664 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:27.498304 (+ 640us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:27.498306 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:27.498653 (+ 347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:27.498654 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:27.498692 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:27.499231 (+ 539us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:47:27.499236 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:47:27.501114 (+ 1878us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.253.254:34179: Server oryx/127.11.253.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":77,"threads_started":1}
W20260806 03:47:27.501281 25427 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 03:47:27.496305 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.253.195:33009 (local address 127.11.253.254:34179)
0806 03:47:27.496400 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0806 03:47:27.496402 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:47:27.496411 (+ 9us) server_negotiation.cc:407] Connection header received
0806 03:47:27.496489 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:47:27.496490 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:47:27.496502 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:47:27.496573 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:47:27.497022 (+ 449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:27.497590 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:27.498382 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:27.498586 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:27.498640 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:27.501211 (+ 2571us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.253.195:33009: BlockingRecv error: recv got EOF from 127.11.253.195:33009 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
Aug 06 03:47:28 dist-test-slave-6ntk krb5kdc[23931](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.253.193@KRBTEST.COM for oryx/127.11.253.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 03:47:28 dist-test-slave-6ntk krb5kdc[23931](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.253.193@KRBTEST.COM for oryx/127.11.253.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 03:47:28.239478 25430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:28.229611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:34179 (local address 127.11.253.193:34079)
0806 03:47:28.229762 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0806 03:47:28.229777 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 03:47:28.229830 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:47:28.230064 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:47:28.230066 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:47:28.230242 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:47:28.230296 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:28.230297 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:28.231053 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:28.231054 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:28.231600 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:28.231602 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:28.232099 (+ 497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:28.232101 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:28.232168 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:28.237175 (+ 5007us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:47:28.237182 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:47:28.239368 (+ 2186us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.253.254:34179: Server oryx/127.11.253.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":89,"threads_started":1}
W20260806 03:47:28.239549 25431 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 03:47:28.229707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.253.193:34079 (local address 127.11.253.254:34179)
0806 03:47:28.229837 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0806 03:47:28.229839 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:47:28.229849 (+ 10us) server_negotiation.cc:407] Connection header received
0806 03:47:28.229914 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:47:28.229915 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:47:28.229927 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:47:28.230014 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:47:28.230407 (+ 393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:28.230974 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:28.231706 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:28.231933 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:28.232072 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:28.239469 (+ 7397us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.253.193:34079: BlockingRecv error: recv got EOF from 127.11.253.193:34079 (error 108)
Metrics: {"server-negotiator.queue_time_us":113,"thread_start_us":73,"threads_started":1}
Aug 06 03:47:28 dist-test-slave-6ntk krb5kdc[23931](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.253.194@KRBTEST.COM for oryx/127.11.253.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 03:47:28 dist-test-slave-6ntk krb5kdc[23931](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.253.194@KRBTEST.COM for oryx/127.11.253.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 03:47:28.370837 25437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:28.366048 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:34179 (local address 127.11.253.194:58441)
0806 03:47:28.366197 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0806 03:47:28.366208 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 03:47:28.366300 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:47:28.366462 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:47:28.366464 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:47:28.366659 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:47:28.366707 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:28.366708 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:28.367408 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:28.367409 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:28.367945 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:28.367947 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:28.368320 (+ 373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:28.368321 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:28.368381 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:28.368856 (+ 475us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:47:28.368860 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:47:28.370758 (+ 1898us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.253.254:34179: Server oryx/127.11.253.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":69,"threads_started":1}
W20260806 03:47:28.370951 25431 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 03:47:28.366108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.253.194:58441 (local address 127.11.253.254:34179)
0806 03:47:28.366163 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0806 03:47:28.366165 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:47:28.366280 (+ 115us) server_negotiation.cc:407] Connection header received
0806 03:47:28.366365 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:47:28.366366 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:47:28.366378 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:47:28.366453 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:47:28.366791 (+ 338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:28.367295 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:28.368034 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:28.368236 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:28.368310 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:28.370868 (+ 2558us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.253.194:58441: BlockingRecv error: recv got EOF from 127.11.253.194:58441 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
Aug 06 03:47:28 dist-test-slave-6ntk krb5kdc[23931](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.253.195@KRBTEST.COM for oryx/127.11.253.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 03:47:28 dist-test-slave-6ntk krb5kdc[23931](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.253.195@KRBTEST.COM for oryx/127.11.253.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 03:47:28.507032 25438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:28.501801 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:34179 (local address 127.11.253.195:60359)
0806 03:47:28.502018 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0806 03:47:28.502031 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 03:47:28.502137 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:47:28.502299 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:47:28.502300 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:47:28.502527 (+ 227us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:47:28.502577 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:28.502579 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:28.503287 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:28.503287 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:28.504096 (+ 809us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:28.504098 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:28.504447 (+ 349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:28.504448 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:28.504506 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:28.504973 (+ 467us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:47:28.504977 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:47:28.506970 (+ 1993us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.253.254:34179: Server oryx/127.11.253.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":138,"threads_started":1}
W20260806 03:47:28.507122 25431 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 03:47:28.501891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.253.195:60359 (local address 127.11.253.254:34179)
0806 03:47:28.501986 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0806 03:47:28.501989 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 03:47:28.502118 (+ 129us) server_negotiation.cc:407] Connection header received
0806 03:47:28.502208 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:47:28.502214 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:47:28.502227 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:47:28.502292 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:47:28.502667 (+ 375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:28.503181 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:28.504165 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:28.504362 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:28.504436 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:28.507072 (+ 2636us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.253.195:60359: BlockingRecv error: recv got EOF from 127.11.253.195:60359 (error 108)
Metrics: {"server-negotiator.queue_time_us":30}
Aug 06 03:47:29 dist-test-slave-6ntk krb5kdc[23931](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.253.193@KRBTEST.COM for oryx/127.11.253.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 03:47:29 dist-test-slave-6ntk krb5kdc[23931](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.253.193@KRBTEST.COM for oryx/127.11.253.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 03:47:29.245036 25440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:29.240042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:34179 (local address 127.11.253.193:59369)
0806 03:47:29.240313 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0806 03:47:29.240326 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 03:47:29.240398 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:47:29.240561 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:47:29.240562 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:47:29.240743 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:47:29.240800 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:29.240801 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:29.241464 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:29.241465 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:29.242026 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:29.242028 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:29.242463 (+ 435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:29.242464 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:29.242517 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:29.243013 (+ 496us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:47:29.243018 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:47:29.244934 (+ 1916us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.253.254:34179: Server oryx/127.11.253.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":233,"thread_start_us":121,"threads_started":1}
W20260806 03:47:29.245093 25441 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 03:47:29.240131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.253.193:59369 (local address 127.11.253.254:34179)
0806 03:47:29.240368 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0806 03:47:29.240371 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 03:47:29.240389 (+ 18us) server_negotiation.cc:407] Connection header received
0806 03:47:29.240463 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:47:29.240464 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:47:29.240474 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:47:29.240533 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:47:29.240869 (+ 336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:29.241386 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:29.242102 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:29.242295 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:29.242379 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:29.245023 (+ 2644us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.253.193:59369: BlockingRecv error: recv got EOF from 127.11.253.193:59369 (error 108)
Metrics: {"server-negotiator.queue_time_us":168,"thread_start_us":86,"threads_started":1}
Aug 06 03:47:29 dist-test-slave-6ntk krb5kdc[23931](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.253.194@KRBTEST.COM for oryx/127.11.253.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 03:47:29 dist-test-slave-6ntk krb5kdc[23931](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.253.194@KRBTEST.COM for oryx/127.11.253.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 03:47:29.375869 25442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:29.371340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:34179 (local address 127.11.253.194:40719)
0806 03:47:29.371497 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0806 03:47:29.371508 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 03:47:29.371565 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:47:29.371732 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:47:29.371733 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:47:29.371900 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:47:29.371970 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:29.371972 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:29.372611 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:29.372612 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:29.373158 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:29.373159 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:29.373507 (+ 348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:29.373508 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:29.373551 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:29.374027 (+ 476us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:47:29.374032 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:47:29.375808 (+ 1776us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.253.254:34179: Server oryx/127.11.253.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":91,"threads_started":1}
W20260806 03:47:29.375990 25441 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 03:47:29.371446 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.253.194:40719 (local address 127.11.253.254:34179)
0806 03:47:29.371519 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0806 03:47:29.371521 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:47:29.371553 (+ 32us) server_negotiation.cc:407] Connection header received
0806 03:47:29.371635 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:47:29.371636 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:47:29.371646 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:47:29.371701 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:47:29.372036 (+ 335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:29.372528 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:29.373235 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:29.373438 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:29.373490 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:29.375912 (+ 2422us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.253.194:40719: BlockingRecv error: recv got EOF from 127.11.253.194:40719 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
Aug 06 03:47:29 dist-test-slave-6ntk krb5kdc[23931](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.253.195@KRBTEST.COM for oryx/127.11.253.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 03:47:29 dist-test-slave-6ntk krb5kdc[23931](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.253.195@KRBTEST.COM for oryx/127.11.253.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 03:47:29.513058 25443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:29.507822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:34179 (local address 127.11.253.195:33479)
0806 03:47:29.508046 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0806 03:47:29.508056 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 03:47:29.508125 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:47:29.508326 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:47:29.508328 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:47:29.508510 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:47:29.508563 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:29.508565 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:29.509452 (+ 887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:29.509454 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:29.510125 (+ 671us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:29.510127 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:29.510492 (+ 365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:29.510493 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:29.510540 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:29.511025 (+ 485us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:47:29.511030 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:47:29.512997 (+ 1967us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.253.254:34179: Server oryx/127.11.253.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":141,"threads_started":1}
W20260806 03:47:29.513226 25441 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 03:47:29.507925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.253.195:33479 (local address 127.11.253.254:34179)
0806 03:47:29.507994 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0806 03:47:29.507996 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:47:29.508122 (+ 126us) server_negotiation.cc:407] Connection header received
0806 03:47:29.508192 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:47:29.508193 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:47:29.508206 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:47:29.508264 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:47:29.508648 (+ 384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:29.509356 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:29.510206 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:29.510409 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:29.510477 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:29.513133 (+ 2656us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.253.195:33479: BlockingRecv error: recv got EOF from 127.11.253.195:33479 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
Aug 06 03:47:30 dist-test-slave-6ntk krb5kdc[23931](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.253.193@KRBTEST.COM for oryx/127.11.253.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 03:47:30 dist-test-slave-6ntk krb5kdc[23931](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.253.193@KRBTEST.COM for oryx/127.11.253.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 03:47:30.250204 25447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:30.245553 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:34179 (local address 127.11.253.193:54509)
0806 03:47:30.245697 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0806 03:47:30.245709 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 03:47:30.245767 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:47:30.245951 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:47:30.245952 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:47:30.246122 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:47:30.246176 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:30.246177 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:30.246852 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:30.246852 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:30.247395 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:30.247397 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:30.247721 (+ 324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:30.247722 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:30.247763 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:30.248223 (+ 460us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:47:30.248227 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:47:30.250140 (+ 1913us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.253.254:34179: Server oryx/127.11.253.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":83,"threads_started":1}
W20260806 03:47:30.250329 25448 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 03:47:30.245644 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.253.193:54509 (local address 127.11.253.254:34179)
0806 03:47:30.245783 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0806 03:47:30.245785 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:47:30.245795 (+ 10us) server_negotiation.cc:407] Connection header received
0806 03:47:30.245833 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:47:30.245833 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:47:30.245843 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:47:30.245922 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:47:30.246241 (+ 319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:30.246773 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:30.247466 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:30.247656 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:30.247705 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:30.250242 (+ 2537us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.253.193:54509: BlockingRecv error: recv got EOF from 127.11.253.193:54509 (error 108)
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":91,"threads_started":1}
Aug 06 03:47:30 dist-test-slave-6ntk krb5kdc[23931](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.253.194@KRBTEST.COM for oryx/127.11.253.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 03:47:30 dist-test-slave-6ntk krb5kdc[23931](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.253.194@KRBTEST.COM for oryx/127.11.253.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 03:47:30.380895 25451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:30.376457 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:34179 (local address 127.11.253.194:39005)
0806 03:47:30.376618 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0806 03:47:30.376629 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 03:47:30.376682 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:47:30.376811 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:47:30.376812 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:47:30.376983 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:47:30.377035 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:30.377037 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:30.377661 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:30.377661 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:30.378239 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:30.378241 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:30.378569 (+ 328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:30.378569 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:30.378608 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:30.379072 (+ 464us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:47:30.379076 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:47:30.380833 (+ 1757us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.253.254:34179: Server oryx/127.11.253.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":99,"threads_started":1}
W20260806 03:47:30.381032 25448 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 03:47:30.376577 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.253.194:39005 (local address 127.11.253.254:34179)
0806 03:47:30.376626 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0806 03:47:30.376632 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 03:47:30.376669 (+ 37us) server_negotiation.cc:407] Connection header received
0806 03:47:30.376736 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:47:30.376736 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:47:30.376746 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:47:30.376788 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:47:30.377107 (+ 319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:30.377588 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:30.378317 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:30.378501 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:30.378556 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:30.380950 (+ 2394us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.253.194:39005: BlockingRecv error: recv got EOF from 127.11.253.194:39005 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
Aug 06 03:47:30 dist-test-slave-6ntk krb5kdc[23931](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.253.195@KRBTEST.COM for oryx/127.11.253.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 03:47:30 dist-test-slave-6ntk krb5kdc[23931](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.253.195@KRBTEST.COM for oryx/127.11.253.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 03:47:30.518435 25452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:30.513610 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:34179 (local address 127.11.253.195:33647)
0806 03:47:30.513782 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0806 03:47:30.513792 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 03:47:30.513856 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:47:30.514040 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:47:30.514041 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:47:30.514213 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:47:30.514265 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:30.514266 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:30.514935 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:30.514936 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:30.515626 (+ 690us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:30.515630 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:30.515994 (+ 364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:30.515995 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:30.516050 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:30.516489 (+ 439us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:47:30.516493 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:47:30.518374 (+ 1881us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.253.254:34179: Server oryx/127.11.253.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":106,"threads_started":1}
W20260806 03:47:30.518548 25448 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 03:47:30.513707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.253.195:33647 (local address 127.11.253.254:34179)
0806 03:47:30.513780 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0806 03:47:30.513782 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:47:30.513856 (+ 74us) server_negotiation.cc:407] Connection header received
0806 03:47:30.513961 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:47:30.513962 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:47:30.513974 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:47:30.514034 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:47:30.514361 (+ 327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:30.514852 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:30.515687 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:30.515918 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:30.515987 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:30.518472 (+ 2485us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.253.195:33647: BlockingRecv error: recv got EOF from 127.11.253.195:33647 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
Aug 06 03:47:31 dist-test-slave-6ntk krb5kdc[23931](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.253.193@KRBTEST.COM for oryx/127.11.253.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 03:47:31 dist-test-slave-6ntk krb5kdc[23931](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.253.193@KRBTEST.COM for oryx/127.11.253.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 03:47:31.263095 25453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:31.250811 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:34179 (local address 127.11.253.193:32973)
0806 03:47:31.251000 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0806 03:47:31.251010 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 03:47:31.251070 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:47:31.251211 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:47:31.251212 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:47:31.251401 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:47:31.251456 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:31.251458 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:31.252211 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:31.252212 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:31.252763 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:31.252765 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:31.253249 (+ 484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:31.253251 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:31.253299 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:31.260968 (+ 7669us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:47:31.260974 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:47:31.263032 (+ 2058us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.253.254:34179: Server oryx/127.11.253.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":128,"threads_started":1}
W20260806 03:47:31.263286 25455 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 03:47:31.250898 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.253.193:32973 (local address 127.11.253.254:34179)
0806 03:47:31.251052 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0806 03:47:31.251053 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 03:47:31.251062 (+ 9us) server_negotiation.cc:407] Connection header received
0806 03:47:31.251117 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:47:31.251118 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:47:31.251129 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:47:31.251193 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:47:31.251565 (+ 372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:31.252133 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:31.252888 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:31.253119 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:31.253209 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:31.263199 (+ 9990us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.253.193:32973: BlockingRecv error: recv got EOF from 127.11.253.193:32973 (error 108)
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":98,"threads_started":1}
Aug 06 03:47:31 dist-test-slave-6ntk krb5kdc[23931](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.253.194@KRBTEST.COM for oryx/127.11.253.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 03:47:31 dist-test-slave-6ntk krb5kdc[23931](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.253.194@KRBTEST.COM for oryx/127.11.253.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 03:47:31.386507 25456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:31.381570 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:34179 (local address 127.11.253.194:42257)
0806 03:47:31.381720 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0806 03:47:31.381731 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 03:47:31.381797 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:47:31.382055 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:47:31.382056 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:47:31.382232 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:47:31.382284 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:31.382285 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:31.382921 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:31.382921 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:31.383444 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:31.383445 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:31.383759 (+ 314us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:31.383759 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:31.383800 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:31.384329 (+ 529us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:47:31.384334 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:47:31.386444 (+ 2110us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.253.254:34179: Server oryx/127.11.253.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":92,"threads_started":1}
W20260806 03:47:31.386605 25455 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 03:47:31.381629 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.253.194:42257 (local address 127.11.253.254:34179)
0806 03:47:31.381705 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0806 03:47:31.381706 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 03:47:31.381799 (+ 93us) server_negotiation.cc:407] Connection header received
0806 03:47:31.381923 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:47:31.381924 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:47:31.381944 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:47:31.381968 (+ 24us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:47:31.382349 (+ 381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:31.382837 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:31.383513 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:31.383696 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:31.383744 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:31.386534 (+ 2790us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.253.194:42257: BlockingRecv error: recv got EOF from 127.11.253.194:42257 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
Aug 06 03:47:31 dist-test-slave-6ntk krb5kdc[23931](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.253.195@KRBTEST.COM for oryx/127.11.253.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 03:47:31 dist-test-slave-6ntk krb5kdc[23931](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.253.195@KRBTEST.COM for oryx/127.11.253.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 03:47:31.524020 25457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:31.518948 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:34179 (local address 127.11.253.195:36213)
0806 03:47:31.519174 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0806 03:47:31.519187 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 03:47:31.519294 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:47:31.519513 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:47:31.519514 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:47:31.519714 (+ 200us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:47:31.519762 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:31.519764 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:31.520449 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:31.520450 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:31.520993 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:31.520995 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:31.521365 (+ 370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:31.521366 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:31.521426 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:31.521970 (+ 544us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:47:31.521974 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:47:31.523958 (+ 1984us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.253.254:34179: Server oryx/127.11.253.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":111,"threads_started":1}
W20260806 03:47:31.524148 25455 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 03:47:31.519048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.253.195:36213 (local address 127.11.253.254:34179)
0806 03:47:31.519157 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0806 03:47:31.519159 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:47:31.519274 (+ 115us) server_negotiation.cc:407] Connection header received
0806 03:47:31.519407 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:47:31.519408 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:47:31.519423 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:47:31.519505 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:47:31.519857 (+ 352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:31.520356 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:31.521082 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:31.521274 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:31.521355 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:31.524064 (+ 2709us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.253.195:36213: BlockingRecv error: recv got EOF from 127.11.253.195:36213 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
Aug 06 03:47:32 dist-test-slave-6ntk krb5kdc[23931](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.253.193@KRBTEST.COM for oryx/127.11.253.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 03:47:32 dist-test-slave-6ntk krb5kdc[23931](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.253.193@KRBTEST.COM for oryx/127.11.253.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 03:47:32.268476 25460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:32.263699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:34179 (local address 127.11.253.193:41999)
0806 03:47:32.263870 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0806 03:47:32.263881 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 03:47:32.263936 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:47:32.264091 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:47:32.264091 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:47:32.264271 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:47:32.264324 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:32.264326 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:32.264982 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:32.264982 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:32.265599 (+ 617us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:32.265601 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:32.266003 (+ 402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:32.266004 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:32.266046 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:32.266619 (+ 573us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:47:32.266623 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:47:32.268408 (+ 1785us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.253.254:34179: Server oryx/127.11.253.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":108,"threads_started":1}
W20260806 03:47:32.268616 25463 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 03:47:32.263788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.253.193:41999 (local address 127.11.253.254:34179)
0806 03:47:32.263905 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0806 03:47:32.263906 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 03:47:32.263919 (+ 13us) server_negotiation.cc:407] Connection header received
0806 03:47:32.263996 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:47:32.263997 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:47:32.264007 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:47:32.264066 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:47:32.264393 (+ 327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:32.264923 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:32.265673 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:32.265911 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:32.265981 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:32.268505 (+ 2524us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.253.193:41999: BlockingRecv error: recv got EOF from 127.11.253.193:41999 (error 108)
Metrics: {"server-negotiator.queue_time_us":87,"thread_start_us":54,"threads_started":1}
Aug 06 03:47:32 dist-test-slave-6ntk krb5kdc[23931](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.253.194@KRBTEST.COM for oryx/127.11.253.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 03:47:32 dist-test-slave-6ntk krb5kdc[23931](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.253.194@KRBTEST.COM for oryx/127.11.253.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 03:47:32.391937 25464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:32.387116 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:34179 (local address 127.11.253.194:35757)
0806 03:47:32.387265 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0806 03:47:32.387275 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 03:47:32.387333 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:47:32.387522 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:47:32.387523 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:47:32.387696 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:47:32.387747 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:32.387749 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:32.388400 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:32.388401 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:32.388933 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:32.388934 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:32.389267 (+ 333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:32.389268 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:32.389307 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:32.389855 (+ 548us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:47:32.389860 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:47:32.391867 (+ 2007us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.253.254:34179: Server oryx/127.11.253.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":73,"threads_started":1}
W20260806 03:47:32.392000 25463 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 03:47:32.387190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.253.194:35757 (local address 127.11.253.254:34179)
0806 03:47:32.387272 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0806 03:47:32.387274 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:47:32.387318 (+ 44us) server_negotiation.cc:407] Connection header received
0806 03:47:32.387431 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:47:32.387431 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:47:32.387443 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:47:32.387499 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:47:32.387816 (+ 317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:32.388321 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:32.389009 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:32.389200 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:32.389254 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:32.391952 (+ 2698us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.253.194:35757: BlockingRecv error: recv got EOF from 127.11.253.194:35757 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Aug 06 03:47:32 dist-test-slave-6ntk krb5kdc[23931](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.253.195@KRBTEST.COM for oryx/127.11.253.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 03:47:32 dist-test-slave-6ntk krb5kdc[23931](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.253.195@KRBTEST.COM for oryx/127.11.253.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 03:47:32.529702 25466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:32.524613 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:34179 (local address 127.11.253.195:46885)
0806 03:47:32.524762 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0806 03:47:32.524781 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 03:47:32.524849 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:47:32.524983 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:47:32.524983 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:47:32.525165 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:47:32.525218 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:32.525220 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:32.525947 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:32.525949 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:32.526507 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:32.526509 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:32.526880 (+ 371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:32.526880 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:32.526923 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:32.527487 (+ 564us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:47:32.527492 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:47:32.529599 (+ 2107us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.253.254:34179: Server oryx/127.11.253.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":93,"threads_started":1}
W20260806 03:47:32.529753 25463 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 03:47:32.524691 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.253.195:46885 (local address 127.11.253.254:34179)
0806 03:47:32.524783 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0806 03:47:32.524784 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 03:47:32.524838 (+ 54us) server_negotiation.cc:407] Connection header received
0806 03:47:32.524894 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:47:32.524895 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:47:32.524908 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:47:32.524966 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:47:32.525291 (+ 325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:32.525789 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:32.526590 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:32.526793 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:32.526867 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:32.529685 (+ 2818us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.253.195:46885: BlockingRecv error: recv got EOF from 127.11.253.195:46885 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Aug 06 03:47:33 dist-test-slave-6ntk krb5kdc[23931](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.253.193@KRBTEST.COM for oryx/127.11.253.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 03:47:33 dist-test-slave-6ntk krb5kdc[23931](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.253.193@KRBTEST.COM for oryx/127.11.253.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 03:47:33.278988 25468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:33.268964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:34179 (local address 127.11.253.193:58013)
0806 03:47:33.269174 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0806 03:47:33.269189 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 03:47:33.269263 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:47:33.269409 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:47:33.269410 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:47:33.269586 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:47:33.269637 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:33.269639 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:33.270494 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:33.270495 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:33.271033 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:33.271035 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:33.271458 (+ 423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:33.271459 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:33.271502 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:33.276970 (+ 5468us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:47:33.276976 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:47:33.278915 (+ 1939us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.253.254:34179: Server oryx/127.11.253.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":74,"threads_started":1}
W20260806 03:47:33.279145 25471 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 03:47:33.269130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.253.193:58013 (local address 127.11.253.254:34179)
0806 03:47:33.269271 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0806 03:47:33.269273 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:47:33.269284 (+ 11us) server_negotiation.cc:407] Connection header received
0806 03:47:33.269309 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:47:33.269310 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:47:33.269320 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:47:33.269387 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:47:33.269704 (+ 317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:33.270418 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:33.271116 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:33.271393 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:33.271450 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:33.279047 (+ 7597us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.253.193:58013: BlockingRecv error: recv got EOF from 127.11.253.193:58013 (error 108)
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":77,"threads_started":1}
Aug 06 03:47:33 dist-test-slave-6ntk krb5kdc[23931](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.253.194@KRBTEST.COM for oryx/127.11.253.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 03:47:33 dist-test-slave-6ntk krb5kdc[23931](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.253.194@KRBTEST.COM for oryx/127.11.253.254@KRBTEST.COM, Server not found in Kerberos database
W20260806 03:47:33.397946 25471 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 03:47:33.392659 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.253.194:42343 (local address 127.11.253.254:34179)
0806 03:47:33.392725 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0806 03:47:33.392727 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:47:33.392846 (+ 119us) server_negotiation.cc:407] Connection header received
0806 03:47:33.392874 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:47:33.392875 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:47:33.392887 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:47:33.392928 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:47:33.393232 (+ 304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:33.393904 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:33.394863 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:33.395077 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:33.395159 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:33.397890 (+ 2731us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.253.194:42343: BlockingRecv error: recv got EOF from 127.11.253.194:42343 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
I20260806 03:47:33.397943 25472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:33.392580 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:34179 (local address 127.11.253.194:42343)
0806 03:47:33.392738 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0806 03:47:33.392748 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 03:47:33.392808 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:47:33.392953 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:47:33.392953 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:47:33.393119 (+ 166us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:47:33.393171 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:33.393172 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:33.393961 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:33.393962 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:33.394752 (+ 790us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:33.394755 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:33.395159 (+ 404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:33.395160 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:33.395216 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:33.395758 (+ 542us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:47:33.395764 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:47:33.397778 (+ 2014us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.253.254:34179: Server oryx/127.11.253.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":91,"threads_started":1}
Aug 06 03:47:33 dist-test-slave-6ntk krb5kdc[23931](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.253.195@KRBTEST.COM for oryx/127.11.253.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 03:47:33 dist-test-slave-6ntk krb5kdc[23931](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.253.195@KRBTEST.COM for oryx/127.11.253.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 03:47:33.535262 25473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:33.530276 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:34179 (local address 127.11.253.195:49227)
0806 03:47:33.530463 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0806 03:47:33.530482 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 03:47:33.530568 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:47:33.530718 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:47:33.530719 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:47:33.530910 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:47:33.530960 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:33.530962 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:33.531655 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:33.531656 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:33.532192 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:33.532194 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:33.532565 (+ 371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:33.532567 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:33.532633 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:33.533186 (+ 553us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:47:33.533191 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:47:33.535199 (+ 2008us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.253.254:34179: Server oryx/127.11.253.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":125,"threads_started":1}
W20260806 03:47:33.535408 25471 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 03:47:33.530475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.253.195:49227 (local address 127.11.253.254:34179)
0806 03:47:33.530545 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0806 03:47:33.530547 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:47:33.530559 (+ 12us) server_negotiation.cc:407] Connection header received
0806 03:47:33.530636 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:47:33.530637 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:47:33.530652 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:47:33.530705 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:47:33.531052 (+ 347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:33.531556 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:33.532277 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:33.532480 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:33.532555 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:33.535326 (+ 2771us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.253.195:49227: BlockingRecv error: recv got EOF from 127.11.253.195:49227 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
Aug 06 03:47:34 dist-test-slave-6ntk krb5kdc[23931](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.253.193@KRBTEST.COM for oryx/127.11.253.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 03:47:34 dist-test-slave-6ntk krb5kdc[23931](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.253.193@KRBTEST.COM for oryx/127.11.253.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 03:47:34.285004 25474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:34.279594 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:34179 (local address 127.11.253.193:54539)
0806 03:47:34.279833 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0806 03:47:34.279857 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0806 03:47:34.279928 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:47:34.280166 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:47:34.280167 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:47:34.280468 (+ 301us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:47:34.280549 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:34.280553 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:34.281419 (+ 866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:34.281420 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:34.281995 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:34.281997 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:34.282345 (+ 348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:34.282346 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:34.282388 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:34.282884 (+ 496us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:47:34.282888 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:47:34.284941 (+ 2053us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.253.254:34179: Server oryx/127.11.253.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":126,"threads_started":1}
W20260806 03:47:34.285290 25475 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 03:47:34.279719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.253.193:54539 (local address 127.11.253.254:34179)
0806 03:47:34.279893 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0806 03:47:34.279896 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 03:47:34.279912 (+ 16us) server_negotiation.cc:407] Connection header received
0806 03:47:34.280015 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:47:34.280016 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:47:34.280029 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:47:34.280130 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:47:34.280657 (+ 527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:34.281315 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:34.282075 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:34.282273 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:34.282334 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:34.285140 (+ 2806us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.253.193:54539: BlockingRecv error: recv got EOF from 127.11.253.193:54539 (error 108)
Metrics: {"server-negotiator.queue_time_us":116,"thread_start_us":91,"threads_started":1}
Aug 06 03:47:34 dist-test-slave-6ntk krb5kdc[23931](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.253.194@KRBTEST.COM for oryx/127.11.253.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 03:47:34 dist-test-slave-6ntk krb5kdc[23931](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.253.194@KRBTEST.COM for oryx/127.11.253.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 03:47:34.403750 25477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:34.398512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:34179 (local address 127.11.253.194:50467)
0806 03:47:34.398672 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0806 03:47:34.398682 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 03:47:34.398733 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:47:34.398919 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:47:34.398920 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:47:34.399124 (+ 204us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:47:34.399191 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:34.399193 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:34.399827 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:34.399828 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:34.400485 (+ 657us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:34.400487 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:34.400930 (+ 443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:34.400931 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:34.400977 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:34.401432 (+ 455us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:47:34.401437 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:47:34.403636 (+ 2199us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.253.254:34179: Server oryx/127.11.253.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":95,"threads_started":1}
W20260806 03:47:34.403784 25475 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 03:47:34.398601 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.253.194:50467 (local address 127.11.253.254:34179)
0806 03:47:34.398764 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0806 03:47:34.398766 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:47:34.398784 (+ 18us) server_negotiation.cc:407] Connection header received
0806 03:47:34.398809 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:47:34.398809 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:47:34.398819 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:47:34.398896 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:47:34.399266 (+ 370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:34.399754 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:34.400545 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:34.400856 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:34.400915 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:34.403717 (+ 2802us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.253.194:50467: BlockingRecv error: recv got EOF from 127.11.253.194:50467 (error 108)
Metrics: {"server-negotiator.queue_time_us":136}
Aug 06 03:47:34 dist-test-slave-6ntk krb5kdc[23931](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.253.195@KRBTEST.COM for oryx/127.11.253.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 03:47:34 dist-test-slave-6ntk krb5kdc[23931](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.253.195@KRBTEST.COM for oryx/127.11.253.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 03:47:34.540498 25478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:34.535844 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:34179 (local address 127.11.253.195:43425)
0806 03:47:34.536009 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0806 03:47:34.536025 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 03:47:34.536086 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:47:34.536245 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:47:34.536245 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:47:34.536429 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:47:34.536478 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:34.536479 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:34.537119 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:34.537119 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:34.537646 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:34.537648 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:34.538022 (+ 374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:34.538023 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:34.538065 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:34.538564 (+ 499us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:47:34.538568 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:47:34.540425 (+ 1857us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.253.254:34179: Server oryx/127.11.253.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":98,"threads_started":1}
W20260806 03:47:34.540598 25475 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 03:47:34.535943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.253.195:43425 (local address 127.11.253.254:34179)
0806 03:47:34.536026 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0806 03:47:34.536028 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:47:34.536111 (+ 83us) server_negotiation.cc:407] Connection header received
0806 03:47:34.536143 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:47:34.536144 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:47:34.536158 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:47:34.536225 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:47:34.536546 (+ 321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:34.537035 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:34.537724 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:34.537957 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:34.538019 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:34.540527 (+ 2508us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.253.195:43425: BlockingRecv error: recv got EOF from 127.11.253.195:43425 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Aug 06 03:47:35 dist-test-slave-6ntk krb5kdc[23931](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.253.193@KRBTEST.COM for oryx/127.11.253.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 03:47:35 dist-test-slave-6ntk krb5kdc[23931](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.253.193@KRBTEST.COM for oryx/127.11.253.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 03:47:35.290728 25479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:35.285604 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:34179 (local address 127.11.253.193:52931)
0806 03:47:35.285834 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0806 03:47:35.285845 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 03:47:35.285917 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:47:35.286090 (+ 173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:47:35.286091 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:47:35.286269 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:47:35.286362 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:35.286364 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:35.287092 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:35.287093 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:35.287653 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:35.287655 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:35.288054 (+ 399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:35.288055 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:35.288096 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:35.288603 (+ 507us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:47:35.288608 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:47:35.290657 (+ 2049us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.253.254:34179: Server oryx/127.11.253.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":158,"threads_started":1}
W20260806 03:47:35.290892 25480 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 03:47:35.285728 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.253.193:52931 (local address 127.11.253.254:34179)
0806 03:47:35.285858 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0806 03:47:35.285860 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:47:35.285902 (+ 42us) server_negotiation.cc:407] Connection header received
0806 03:47:35.285996 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:47:35.285997 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:47:35.286008 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:47:35.286074 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:47:35.286438 (+ 364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:35.287017 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:35.287755 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:35.287980 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:35.288044 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:35.290785 (+ 2741us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.253.193:52931: BlockingRecv error: recv got EOF from 127.11.253.193:52931 (error 108)
Metrics: {"server-negotiator.queue_time_us":104,"thread_start_us":64,"threads_started":1}
Aug 06 03:47:35 dist-test-slave-6ntk krb5kdc[23931](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.253.194@KRBTEST.COM for oryx/127.11.253.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 03:47:35 dist-test-slave-6ntk krb5kdc[23931](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.253.194@KRBTEST.COM for oryx/127.11.253.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 03:47:35.409510 25481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:35.404308 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:34179 (local address 127.11.253.194:59787)
0806 03:47:35.404556 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0806 03:47:35.404569 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 03:47:35.404667 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:47:35.404843 (+ 176us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:47:35.404843 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:47:35.405056 (+ 213us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:47:35.405105 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:35.405107 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:35.406052 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:35.406053 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:35.406589 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:35.406590 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:35.406978 (+ 388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:35.406979 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:35.407036 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:35.407496 (+ 460us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:47:35.407501 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:47:35.409446 (+ 1945us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.253.254:34179: Server oryx/127.11.253.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":227,"thread_start_us":129,"threads_started":1}
W20260806 03:47:35.409636 25480 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 03:47:35.404452 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.253.194:59787 (local address 127.11.253.254:34179)
0806 03:47:35.404535 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0806 03:47:35.404537 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:47:35.404644 (+ 107us) server_negotiation.cc:407] Connection header received
0806 03:47:35.404741 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:47:35.404742 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:47:35.404755 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:47:35.404834 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:47:35.405195 (+ 361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:35.405986 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:35.406689 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:35.406902 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:35.406980 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:35.409553 (+ 2573us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.253.194:59787: BlockingRecv error: recv got EOF from 127.11.253.194:59787 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Aug 06 03:47:35 dist-test-slave-6ntk krb5kdc[23931](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.253.195@KRBTEST.COM for oryx/127.11.253.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 03:47:35 dist-test-slave-6ntk krb5kdc[23931](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.253.195@KRBTEST.COM for oryx/127.11.253.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 03:47:35.545949 25483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:35.541065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:34179 (local address 127.11.253.195:35397)
0806 03:47:35.541303 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0806 03:47:35.541314 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 03:47:35.541356 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:47:35.541485 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:47:35.541486 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:47:35.541667 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:47:35.541724 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:35.541726 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:35.542463 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:35.542464 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:35.542994 (+ 530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:35.542996 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:35.543457 (+ 461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:35.543458 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:35.543501 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:35.544009 (+ 508us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:47:35.544014 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:47:35.545820 (+ 1806us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.253.254:34179: Server oryx/127.11.253.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":220,"thread_start_us":144,"threads_started":1}
W20260806 03:47:35.546078 25480 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 03:47:35.541250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.253.195:35397 (local address 127.11.253.254:34179)
0806 03:47:35.541329 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0806 03:47:35.541331 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:47:35.541364 (+ 33us) server_negotiation.cc:407] Connection header received
0806 03:47:35.541412 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:47:35.541413 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:47:35.541425 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:47:35.541465 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:47:35.541799 (+ 334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:35.542380 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:35.543110 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:35.543350 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:35.543422 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:35.545977 (+ 2555us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.253.195:35397: BlockingRecv error: recv got EOF from 127.11.253.195:35397 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Aug 06 03:47:36 dist-test-slave-6ntk krb5kdc[23931](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.253.193@KRBTEST.COM for oryx/127.11.253.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 03:47:36 dist-test-slave-6ntk krb5kdc[23931](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.253.193@KRBTEST.COM for oryx/127.11.253.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 03:47:36.296391 25484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:36.291362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:34179 (local address 127.11.253.193:41633)
0806 03:47:36.291545 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0806 03:47:36.291557 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 03:47:36.291624 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:47:36.291801 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:47:36.291802 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:47:36.291998 (+ 196us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:47:36.292050 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:36.292051 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:36.292775 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:36.292776 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:36.293550 (+ 774us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:36.293553 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:36.293946 (+ 393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:36.293947 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:36.294009 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:36.294508 (+ 499us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:47:36.294512 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:47:36.296290 (+ 1778us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.253.254:34179: Server oryx/127.11.253.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":109,"threads_started":1}
W20260806 03:47:36.296504 25486 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 03:47:36.291486 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.253.193:41633 (local address 127.11.253.254:34179)
0806 03:47:36.291636 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0806 03:47:36.291638 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:47:36.291683 (+ 45us) server_negotiation.cc:407] Connection header received
0806 03:47:36.291709 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:47:36.291709 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:47:36.291721 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:47:36.291775 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:47:36.292132 (+ 357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:36.292657 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:36.293620 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:36.293823 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:36.293901 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:36.296423 (+ 2522us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.253.193:41633: BlockingRecv error: recv got EOF from 127.11.253.193:41633 (error 108)
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":93,"threads_started":1}
Aug 06 03:47:36 dist-test-slave-6ntk krb5kdc[23931](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.253.194@KRBTEST.COM for oryx/127.11.253.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 03:47:36 dist-test-slave-6ntk krb5kdc[23931](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.253.194@KRBTEST.COM for oryx/127.11.253.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 03:47:36.414745 25488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:36.410058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:34179 (local address 127.11.253.194:57989)
0806 03:47:36.410217 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0806 03:47:36.410233 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 03:47:36.410289 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:47:36.410435 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:47:36.410435 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:47:36.410608 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:47:36.410657 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:36.410658 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:36.411307 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:36.411308 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:36.411881 (+ 573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:36.411883 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:36.412199 (+ 316us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:36.412200 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:36.412256 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:36.412834 (+ 578us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:47:36.412839 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:47:36.414679 (+ 1840us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.253.254:34179: Server oryx/127.11.253.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":105,"threads_started":1}
W20260806 03:47:36.414850 25486 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 03:47:36.410132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.253.194:57989 (local address 127.11.253.254:34179)
0806 03:47:36.410185 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0806 03:47:36.410187 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:47:36.410276 (+ 89us) server_negotiation.cc:407] Connection header received
0806 03:47:36.410354 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:47:36.410354 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:47:36.410364 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:47:36.410418 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:47:36.410723 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:36.411230 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:36.411955 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:36.412135 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:36.412185 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:36.414776 (+ 2591us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.253.194:57989: BlockingRecv error: recv got EOF from 127.11.253.194:57989 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
Aug 06 03:47:36 dist-test-slave-6ntk krb5kdc[23931](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.253.195@KRBTEST.COM for oryx/127.11.253.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 03:47:36 dist-test-slave-6ntk krb5kdc[23931](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.253.195@KRBTEST.COM for oryx/127.11.253.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 03:47:36.552037 25489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:36.546535 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:34179 (local address 127.11.253.195:38007)
0806 03:47:36.546695 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0806 03:47:36.546706 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 03:47:36.546776 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:47:36.546963 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:47:36.546964 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:47:36.547146 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:47:36.547195 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:36.547197 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:36.547912 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:36.547912 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:36.548482 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:36.548484 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:36.548869 (+ 385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:36.548870 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:36.548925 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:36.549600 (+ 675us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:47:36.549606 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:47:36.551966 (+ 2360us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.253.254:34179: Server oryx/127.11.253.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":74,"threads_started":1}
W20260806 03:47:36.552201 25486 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 03:47:36.546637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.253.195:38007 (local address 127.11.253.254:34179)
0806 03:47:36.546697 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0806 03:47:36.546699 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:47:36.546762 (+ 63us) server_negotiation.cc:407] Connection header received
0806 03:47:36.546837 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:47:36.546838 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:47:36.546855 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:47:36.546903 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:47:36.547301 (+ 398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:36.547828 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:36.548563 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:36.548759 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:36.548815 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:36.552104 (+ 3289us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.253.195:38007: BlockingRecv error: recv got EOF from 127.11.253.195:38007 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
Aug 06 03:47:37 dist-test-slave-6ntk krb5kdc[23931](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.253.193@KRBTEST.COM for oryx/127.11.253.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 03:47:37 dist-test-slave-6ntk krb5kdc[23931](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.253.193@KRBTEST.COM for oryx/127.11.253.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 03:47:37.302204 25490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:37.296990 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:34179 (local address 127.11.253.193:55141)
0806 03:47:37.297249 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0806 03:47:37.297260 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 03:47:37.297330 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:47:37.297482 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:47:37.297483 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:47:37.297665 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:47:37.297716 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:37.297718 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:37.298600 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:37.298602 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:37.299151 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:37.299153 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:37.299592 (+ 439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:37.299593 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:37.299660 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:37.300187 (+ 527us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:47:37.300192 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:47:37.302103 (+ 1911us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.253.254:34179: Server oryx/127.11.253.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":231,"thread_start_us":188,"threads_started":1}
W20260806 03:47:37.302254 25491 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 03:47:37.297089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.253.193:55141 (local address 127.11.253.254:34179)
0806 03:47:37.297225 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0806 03:47:37.297227 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:47:37.297322 (+ 95us) server_negotiation.cc:407] Connection header received
0806 03:47:37.297378 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:47:37.297379 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:47:37.297391 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:47:37.297457 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:47:37.297827 (+ 370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:37.298495 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:37.299295 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:37.299512 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:37.299591 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:37.302182 (+ 2591us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.253.193:55141: BlockingRecv error: recv got EOF from 127.11.253.193:55141 (error 108)
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":78,"threads_started":1}
Aug 06 03:47:37 dist-test-slave-6ntk krb5kdc[23931](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.253.194@KRBTEST.COM for oryx/127.11.253.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 03:47:37 dist-test-slave-6ntk krb5kdc[23931](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.253.194@KRBTEST.COM for oryx/127.11.253.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 03:47:37.420517 25494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:37.415262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:34179 (local address 127.11.253.194:43877)
0806 03:47:37.415467 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0806 03:47:37.415482 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 03:47:37.415585 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:47:37.415736 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:47:37.415737 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:47:37.415958 (+ 221us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:47:37.416004 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:37.416006 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:37.416705 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:37.416706 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:37.417256 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:37.417258 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:37.417694 (+ 436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:37.417695 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:37.417777 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:37.418407 (+ 630us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:47:37.418412 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:47:37.420406 (+ 1994us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.253.254:34179: Server oryx/127.11.253.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":89,"threads_started":1}
W20260806 03:47:37.420554 25491 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 03:47:37.415372 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.253.194:43877 (local address 127.11.253.254:34179)
0806 03:47:37.415472 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0806 03:47:37.415475 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 03:47:37.415564 (+ 89us) server_negotiation.cc:407] Connection header received
0806 03:47:37.415644 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:47:37.415645 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:47:37.415658 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:47:37.415730 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:47:37.416090 (+ 360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:37.416594 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:37.417347 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:37.417591 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:37.417669 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:37.420489 (+ 2820us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.253.194:43877: BlockingRecv error: recv got EOF from 127.11.253.194:43877 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
Aug 06 03:47:37 dist-test-slave-6ntk krb5kdc[23931](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.253.195@KRBTEST.COM for oryx/127.11.253.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 03:47:37 dist-test-slave-6ntk krb5kdc[23931](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.253.195@KRBTEST.COM for oryx/127.11.253.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 03:47:37.557337 25496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:37.552542 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:34179 (local address 127.11.253.195:34673)
0806 03:47:37.552701 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0806 03:47:37.552712 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 03:47:37.552772 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:47:37.553014 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:47:37.553015 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:47:37.553193 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:47:37.553242 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:37.553244 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:37.553955 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:37.553956 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:37.554493 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:37.554495 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:37.554850 (+ 355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:37.554851 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:37.554890 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:37.555395 (+ 505us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:47:37.555399 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:47:37.557236 (+ 1837us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.253.254:34179: Server oryx/127.11.253.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":97,"threads_started":1}
W20260806 03:47:37.557368 25491 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 03:47:37.552610 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.253.195:34673 (local address 127.11.253.254:34179)
0806 03:47:37.552665 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0806 03:47:37.552667 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:47:37.552830 (+ 163us) server_negotiation.cc:407] Connection header received
0806 03:47:37.552900 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:47:37.552901 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:47:37.552920 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:47:37.552994 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:47:37.553319 (+ 325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:37.553814 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:37.554587 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:37.554779 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:37.554839 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:37.557315 (+ 2476us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.253.195:34673: BlockingRecv error: recv got EOF from 127.11.253.195:34673 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
Aug 06 03:47:38 dist-test-slave-6ntk krb5kdc[23931](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.253.193@KRBTEST.COM for oryx/127.11.253.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 03:47:38 dist-test-slave-6ntk krb5kdc[23931](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.253.193@KRBTEST.COM for oryx/127.11.253.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 03:47:38.310393 25498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:38.302707 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:34179 (local address 127.11.253.193:56109)
0806 03:47:38.302898 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0806 03:47:38.302910 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 03:47:38.302964 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:47:38.303096 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:47:38.303096 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:47:38.303277 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:47:38.303327 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:38.303329 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:38.303982 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:38.303983 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:38.304550 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:38.304552 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:38.304883 (+ 331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:38.304884 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:38.304927 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:38.308307 (+ 3380us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:47:38.308313 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:47:38.310295 (+ 1982us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.253.254:34179: Server oryx/127.11.253.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":113,"threads_started":1}
W20260806 03:47:38.310510 25499 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 03:47:38.302822 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.253.193:56109 (local address 127.11.253.254:34179)
0806 03:47:38.302958 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0806 03:47:38.302960 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:47:38.302970 (+ 10us) server_negotiation.cc:407] Connection header received
0806 03:47:38.303007 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:47:38.303008 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:47:38.303017 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:47:38.303074 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:47:38.303407 (+ 333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:38.303905 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:38.304624 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:38.304817 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:38.304867 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:38.310435 (+ 5568us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.253.193:56109: BlockingRecv error: recv got EOF from 127.11.253.193:56109 (error 108)
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":80,"threads_started":1}
Aug 06 03:47:38 dist-test-slave-6ntk krb5kdc[23931](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.253.194@KRBTEST.COM for oryx/127.11.253.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 03:47:38 dist-test-slave-6ntk krb5kdc[23931](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.253.194@KRBTEST.COM for oryx/127.11.253.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 03:47:38.425683 25500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:38.421117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:34179 (local address 127.11.253.194:36155)
0806 03:47:38.421293 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0806 03:47:38.421305 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 03:47:38.421421 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:47:38.421592 (+ 171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:47:38.421592 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:47:38.421823 (+ 231us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:47:38.421891 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:38.421894 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:38.422539 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:38.422540 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:38.423063 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:38.423064 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:38.423404 (+ 340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:38.423405 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:38.423448 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:38.423902 (+ 454us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:47:38.423906 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:47:38.425623 (+ 1717us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.253.254:34179: Server oryx/127.11.253.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":103,"threads_started":1}
W20260806 03:47:38.425793 25499 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 03:47:38.421244 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.253.194:36155 (local address 127.11.253.254:34179)
0806 03:47:38.421309 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0806 03:47:38.421312 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 03:47:38.421422 (+ 110us) server_negotiation.cc:407] Connection header received
0806 03:47:38.421485 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:47:38.421486 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:47:38.421499 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:47:38.421555 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:47:38.421970 (+ 415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:38.422464 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:38.423140 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:38.423334 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:38.423388 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:38.425721 (+ 2333us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.253.194:36155: BlockingRecv error: recv got EOF from 127.11.253.194:36155 (error 108)
Metrics: {"server-negotiator.queue_time_us":27}
Aug 06 03:47:38 dist-test-slave-6ntk krb5kdc[23931](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.253.195@KRBTEST.COM for oryx/127.11.253.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 03:47:38 dist-test-slave-6ntk krb5kdc[23931](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.253.195@KRBTEST.COM for oryx/127.11.253.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 03:47:38.563198 25501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:38.558028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:34179 (local address 127.11.253.195:55797)
0806 03:47:38.558261 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0806 03:47:38.558273 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 03:47:38.558396 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:47:38.558570 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:47:38.558571 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:47:38.558770 (+ 199us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:47:38.558820 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:38.558822 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:38.559503 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:38.559504 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:38.560044 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:38.560046 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:38.560414 (+ 368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:38.560415 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:38.560473 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:38.560970 (+ 497us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:47:38.560974 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:47:38.563137 (+ 2163us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.253.254:34179: Server oryx/127.11.253.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":123,"threads_started":1}
W20260806 03:47:38.563403 25499 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 03:47:38.558137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.253.195:55797 (local address 127.11.253.254:34179)
0806 03:47:38.558246 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0806 03:47:38.558249 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 03:47:38.558375 (+ 126us) server_negotiation.cc:407] Connection header received
0806 03:47:38.558470 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:47:38.558471 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:47:38.558485 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:47:38.558561 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:47:38.558910 (+ 349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:38.559413 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:38.560139 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:38.560337 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:38.560416 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:38.563321 (+ 2905us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.253.195:55797: BlockingRecv error: recv got EOF from 127.11.253.195:55797 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
Aug 06 03:47:39 dist-test-slave-6ntk krb5kdc[23931](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.253.193@KRBTEST.COM for oryx/127.11.253.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 03:47:39 dist-test-slave-6ntk krb5kdc[23931](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.253.193@KRBTEST.COM for oryx/127.11.253.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 03:47:39.315570 25502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:39.310947 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:34179 (local address 127.11.253.193:59703)
0806 03:47:39.311117 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0806 03:47:39.311127 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 03:47:39.311184 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:47:39.311351 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:47:39.311352 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:47:39.311525 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:47:39.311575 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:39.311577 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:39.312213 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:39.312213 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:39.312732 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:39.312733 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:39.313086 (+ 353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:39.313087 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:39.313128 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:39.313603 (+ 475us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:47:39.313607 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:47:39.315509 (+ 1902us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.253.254:34179: Server oryx/127.11.253.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":108,"threads_started":1}
W20260806 03:47:39.315712 25503 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 03:47:39.311052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.253.193:59703 (local address 127.11.253.254:34179)
0806 03:47:39.311219 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0806 03:47:39.311221 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:47:39.311232 (+ 11us) server_negotiation.cc:407] Connection header received
0806 03:47:39.311259 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:47:39.311260 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:47:39.311272 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:47:39.311334 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:47:39.311642 (+ 308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:39.312146 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:39.312809 (+ 663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:39.313012 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:39.313069 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:39.315615 (+ 2546us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.253.193:59703: BlockingRecv error: recv got EOF from 127.11.253.193:59703 (error 108)
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":103,"threads_started":1}
Aug 06 03:47:39 dist-test-slave-6ntk krb5kdc[23931](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.253.194@KRBTEST.COM for oryx/127.11.253.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 03:47:39 dist-test-slave-6ntk krb5kdc[23931](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.253.194@KRBTEST.COM for oryx/127.11.253.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 03:47:39.431295 25504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:39.426242 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:34179 (local address 127.11.253.194:59717)
0806 03:47:39.426445 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0806 03:47:39.426457 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 03:47:39.426550 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:47:39.426734 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:47:39.426735 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:47:39.426957 (+ 222us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:47:39.427005 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:39.427007 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:39.427713 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:39.427714 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:39.428365 (+ 651us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:39.428367 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:39.428724 (+ 357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:39.428725 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:39.428781 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:39.429273 (+ 492us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:47:39.429278 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:47:39.431194 (+ 1916us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.253.254:34179: Server oryx/127.11.253.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":111,"threads_started":1}
W20260806 03:47:39.431357 25503 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 03:47:39.426348 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.253.194:59717 (local address 127.11.253.254:34179)
0806 03:47:39.426445 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0806 03:47:39.426447 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:47:39.426529 (+ 82us) server_negotiation.cc:407] Connection header received
0806 03:47:39.426628 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:47:39.426630 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:47:39.426642 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:47:39.426713 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:47:39.427092 (+ 379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:39.427624 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:39.428446 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:39.428650 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:39.428727 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:39.431281 (+ 2554us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.253.194:59717: BlockingRecv error: recv got EOF from 127.11.253.194:59717 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Aug 06 03:47:39 dist-test-slave-6ntk krb5kdc[23931](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.253.195@KRBTEST.COM for oryx/127.11.253.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 03:47:39 dist-test-slave-6ntk krb5kdc[23931](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.253.195@KRBTEST.COM for oryx/127.11.253.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 03:47:39.569134 25507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:39.563691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:34179 (local address 127.11.253.195:44715)
0806 03:47:39.563857 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0806 03:47:39.563869 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 03:47:39.563931 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:47:39.564083 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:47:39.564084 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:47:39.564274 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:47:39.564325 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:39.564327 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:39.565056 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:39.565057 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:39.565637 (+ 580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:39.565639 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:39.566094 (+ 455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:39.566095 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:39.566143 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:39.566757 (+ 614us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:47:39.566762 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:47:39.569035 (+ 2273us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.253.254:34179: Server oryx/127.11.253.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":90,"threads_started":1}
W20260806 03:47:39.569216 25503 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 03:47:39.563774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.253.195:44715 (local address 127.11.253.254:34179)
0806 03:47:39.563887 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0806 03:47:39.563888 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 03:47:39.563934 (+ 46us) server_negotiation.cc:407] Connection header received
0806 03:47:39.563988 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:47:39.563989 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:47:39.564002 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:47:39.564062 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:47:39.564400 (+ 338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:39.564970 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:39.565726 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:39.566022 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:39.566081 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:39.569150 (+ 3069us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.253.195:44715: BlockingRecv error: recv got EOF from 127.11.253.195:44715 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
Aug 06 03:47:40 dist-test-slave-6ntk krb5kdc[23931](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.253.193@KRBTEST.COM for oryx/127.11.253.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 03:47:40 dist-test-slave-6ntk krb5kdc[23931](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.253.193@KRBTEST.COM for oryx/127.11.253.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 03:47:40.321159 25509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:40.316168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:34179 (local address 127.11.253.193:59509)
0806 03:47:40.316439 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0806 03:47:40.316458 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 03:47:40.316518 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:47:40.316662 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:47:40.316662 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:47:40.316837 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:47:40.316890 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:40.316891 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:40.317505 (+ 614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:40.317506 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:40.318096 (+ 590us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:40.318098 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:40.318421 (+ 323us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:40.318422 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:40.318463 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:40.318928 (+ 465us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:47:40.318933 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:47:40.321086 (+ 2153us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.253.254:34179: Server oryx/127.11.253.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":250,"thread_start_us":222,"threads_started":1}
W20260806 03:47:40.321286 25510 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 03:47:40.316250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.253.193:59509 (local address 127.11.253.254:34179)
0806 03:47:40.316388 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0806 03:47:40.316390 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:47:40.316517 (+ 127us) server_negotiation.cc:407] Connection header received
0806 03:47:40.316565 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:47:40.316566 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:47:40.316577 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:47:40.316639 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:47:40.316951 (+ 312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:40.317440 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:40.318165 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:40.318357 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:40.318408 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:40.321179 (+ 2771us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.253.193:59509: BlockingRecv error: recv got EOF from 127.11.253.193:59509 (error 108)
Metrics: {"server-negotiator.queue_time_us":108,"thread_start_us":73,"threads_started":1}
Aug 06 03:47:40 dist-test-slave-6ntk krb5kdc[23931](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.253.194@KRBTEST.COM for oryx/127.11.253.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 03:47:40 dist-test-slave-6ntk krb5kdc[23931](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.253.194@KRBTEST.COM for oryx/127.11.253.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 03:47:40.436681 25511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:40.431892 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:34179 (local address 127.11.253.194:39961)
0806 03:47:40.432046 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0806 03:47:40.432057 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 03:47:40.432123 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:47:40.432248 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:47:40.432248 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:47:40.432419 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:47:40.432468 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:40.432470 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:40.433151 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:40.433152 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:40.433757 (+ 605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:40.433760 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:40.434187 (+ 427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:40.434188 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:40.434271 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:40.434765 (+ 494us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:47:40.434769 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:47:40.436605 (+ 1836us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.253.254:34179: Server oryx/127.11.253.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":94,"threads_started":1}
W20260806 03:47:40.436807 25510 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 03:47:40.431956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.253.194:39961 (local address 127.11.253.254:34179)
0806 03:47:40.432035 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0806 03:47:40.432037 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:47:40.432120 (+ 83us) server_negotiation.cc:407] Connection header received
0806 03:47:40.432171 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:47:40.432172 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:47:40.432182 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:47:40.432232 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:47:40.432545 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:40.433049 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:40.433888 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:40.434083 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:40.434172 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:40.436704 (+ 2532us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.253.194:39961: BlockingRecv error: recv got EOF from 127.11.253.194:39961 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
I20260806 03:47:40.482113 12279 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskToVWRe/build/release/bin/kudu with pid 24776
I20260806 03:47:40.494684 12279 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskToVWRe/build/release/bin/kudu with pid 25001
I20260806 03:47:40.506670 12279 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskToVWRe/build/release/bin/kudu with pid 25209
I20260806 03:47:40.514366 12279 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskToVWRe/build/release/bin/kudu with pid 24673
2026-08-06T03:47:40Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (17788 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-06T03:47:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T03:47:40Z Disabled control of system clock
I20260806 03:47:40.551446 12279 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskToVWRe/build/release/bin/kudu
/tmp/dist-test-taskToVWRe/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785988004637995-12279-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785988004637995-12279-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785988004637995-12279-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785988004637995-12279-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.253.254:43437
--webserver_interface=127.11.253.254
--webserver_port=0
--builtin_ntp_servers=127.11.253.212:37559
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.253.254:43437
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260806 03:47:40.634837 25520 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:47:40.635046 25520 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:47:40.635066 25520 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:47:40.636286 25520 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 03:47:40.636336 25520 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:47:40.636348 25520 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:47:40.636361 25520 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 03:47:40.636374 25520 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 03:47:40.637686 25520 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.253.212:37559
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785988004637995-12279-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785988004637995-12279-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.253.254:43437
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.253.254:43437
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785988004637995-12279-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.253.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25520
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785988004637995-12279-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:47:40.637966 25520 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:47:40.638233 25520 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:47:40.643500 25528 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:47:40.643523 25520 server_base.cc:1034] running on GCE node
W20260806 03:47:40.643556 25530 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:47:40.643507 25533 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:47:40.643981 25520 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:47:40.644279 25520 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:47:40.645430 25520 hybrid_clock.cc:648] HybridClock initialized: now 1785988060645419 us; error 29 us; skew 500 ppm
I20260806 03:47:40.646853 25520 webserver.cc:460] Webserver started at http://127.11.253.254:35497/ using document root <none> and password file <none>
I20260806 03:47:40.647164 25520 fs_manager.cc:359] Metadata directory not provided
I20260806 03:47:40.647270 25520 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:47:40.647486 25520 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:47:40.648701 25520 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785988004637995-12279-0/minicluster-data/master-0/data/instance:
uuid: "cbed6c26d33945a68a23738f85753b63"
format_stamp: "Formatted at 2026-08-06 03:47:40 on dist-test-slave-6ntk"
server_key: "226b5a0ff25cac21a9dae55584dbe94a"
server_key_iv: "5ddc1b3f76f4dbf546114a7db30c36a9"
server_key_version: "encryptionkey@0"
I20260806 03:47:40.649118 25520 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785988004637995-12279-0/minicluster-data/master-0/wal/instance:
uuid: "cbed6c26d33945a68a23738f85753b63"
format_stamp: "Formatted at 2026-08-06 03:47:40 on dist-test-slave-6ntk"
server_key: "226b5a0ff25cac21a9dae55584dbe94a"
server_key_iv: "5ddc1b3f76f4dbf546114a7db30c36a9"
server_key_version: "encryptionkey@0"
I20260806 03:47:40.650677 25520 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.004s sys 0.000s
I20260806 03:47:40.652884 25546 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:47:40.653074 25520 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 03:47:40.653209 25520 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785988004637995-12279-0/minicluster-data/master-0/data,/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785988004637995-12279-0/minicluster-data/master-0/wal
uuid: "cbed6c26d33945a68a23738f85753b63"
format_stamp: "Formatted at 2026-08-06 03:47:40 on dist-test-slave-6ntk"
server_key: "226b5a0ff25cac21a9dae55584dbe94a"
server_key_iv: "5ddc1b3f76f4dbf546114a7db30c36a9"
server_key_version: "encryptionkey@0"
I20260806 03:47:40.653311 25520 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785988004637995-12279-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785988004637995-12279-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785988004637995-12279-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:47:40.684401 25520 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:47:40.684798 25520 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:47:40.685001 25520 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:47:40.689719 25520 rpc_server.cc:307] RPC server started. Bound to: 127.11.253.254:43437
I20260806 03:47:40.689757 25619 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.253.254:43437 every 8 connection(s)
I20260806 03:47:40.690212 25520 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785988004637995-12279-0/minicluster-data/master-0/data/info.pb
I20260806 03:47:40.690938 25620 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:47:40.693691 25620 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cbed6c26d33945a68a23738f85753b63: Bootstrap starting.
I20260806 03:47:40.694324 25620 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cbed6c26d33945a68a23738f85753b63: Neither blocks nor log segments found. Creating new log.
I20260806 03:47:40.694655 25620 log.cc:824] T 00000000000000000000000000000000 P cbed6c26d33945a68a23738f85753b63: Log is configured to *not* fsync() on all Append() calls
I20260806 03:47:40.695395 25620 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cbed6c26d33945a68a23738f85753b63: No bootstrap required, opened a new log
I20260806 03:47:40.696509 12279 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskToVWRe/build/release/bin/kudu as pid 25520
I20260806 03:47:40.696584 12279 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785988004637995-12279-0/minicluster-data/master-0/wal/instance
I20260806 03:47:40.696791 12279 external_mini_cluster.cc:1428] Setting key 08417025d876860b83f0cf7faef1c360
I20260806 03:47:40.697296 25620 raft_consensus.cc:348] T 00000000000000000000000000000000 P cbed6c26d33945a68a23738f85753b63 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cbed6c26d33945a68a23738f85753b63" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 43437 } }
I20260806 03:47:40.697501 25620 raft_consensus.cc:374] T 00000000000000000000000000000000 P cbed6c26d33945a68a23738f85753b63 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:47:40.697549 25620 raft_consensus.cc:729] T 00000000000000000000000000000000 P cbed6c26d33945a68a23738f85753b63 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cbed6c26d33945a68a23738f85753b63, State: Initialized, Role: FOLLOWER
I20260806 03:47:40.697677 25620 consensus_queue.cc:260] T 00000000000000000000000000000000 P cbed6c26d33945a68a23738f85753b63 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cbed6c26d33945a68a23738f85753b63" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 43437 } }
I20260806 03:47:40.697779 25620 raft_consensus.cc:388] T 00000000000000000000000000000000 P cbed6c26d33945a68a23738f85753b63 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 03:47:40.697824 25620 raft_consensus.cc:482] T 00000000000000000000000000000000 P cbed6c26d33945a68a23738f85753b63 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 03:47:40.697906 25620 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cbed6c26d33945a68a23738f85753b63 [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:47:40.698455 25620 raft_consensus.cc:504] T 00000000000000000000000000000000 P cbed6c26d33945a68a23738f85753b63 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cbed6c26d33945a68a23738f85753b63" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 43437 } }
I20260806 03:47:40.698578 25620 leader_election.cc:302] T 00000000000000000000000000000000 P cbed6c26d33945a68a23738f85753b63 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cbed6c26d33945a68a23738f85753b63; no voters:
I20260806 03:47:40.698745 25620 leader_election.cc:288] T 00000000000000000000000000000000 P cbed6c26d33945a68a23738f85753b63 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 03:47:40.698781 25627 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cbed6c26d33945a68a23738f85753b63 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 03:47:40.698910 25627 raft_consensus.cc:686] T 00000000000000000000000000000000 P cbed6c26d33945a68a23738f85753b63 [term 1 LEADER]: Becoming Leader. State: Replica: cbed6c26d33945a68a23738f85753b63, State: Running, Role: LEADER
I20260806 03:47:40.699010 25620 sys_catalog.cc:564] T 00000000000000000000000000000000 P cbed6c26d33945a68a23738f85753b63 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 03:47:40.698992 25627 consensus_queue.cc:237] T 00000000000000000000000000000000 P cbed6c26d33945a68a23738f85753b63 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cbed6c26d33945a68a23738f85753b63" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 43437 } }
I20260806 03:47:40.699339 25627 sys_catalog.cc:455] T 00000000000000000000000000000000 P cbed6c26d33945a68a23738f85753b63 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cbed6c26d33945a68a23738f85753b63. Latest consensus state: current_term: 1 leader_uuid: "cbed6c26d33945a68a23738f85753b63" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cbed6c26d33945a68a23738f85753b63" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 43437 } } }
I20260806 03:47:40.699424 25627 sys_catalog.cc:458] T 00000000000000000000000000000000 P cbed6c26d33945a68a23738f85753b63 [sys.catalog]: This master's current role is: LEADER
I20260806 03:47:40.699352 25629 sys_catalog.cc:455] T 00000000000000000000000000000000 P cbed6c26d33945a68a23738f85753b63 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cbed6c26d33945a68a23738f85753b63" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cbed6c26d33945a68a23738f85753b63" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 43437 } } }
I20260806 03:47:40.699592 25629 sys_catalog.cc:458] T 00000000000000000000000000000000 P cbed6c26d33945a68a23738f85753b63 [sys.catalog]: This master's current role is: LEADER
I20260806 03:47:40.699631 25632 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 03:47:40.700192 25632 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 03:47:40.702504 25632 catalog_manager.cc:1347] Generated new cluster ID: 5150eb0fd1264f689d7d2c264e6c4fb4
I20260806 03:47:40.702564 25632 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 03:47:40.703321 25625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:40.697404 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51118 (local address 127.11.253.254:43437)
0806 03:47:40.697650 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0806 03:47:40.697659 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 03:47:40.697689 (+ 30us) server_negotiation.cc:407] Connection header received
0806 03:47:40.697849 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:47:40.697857 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:47:40.697958 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:47:40.698099 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:47:40.698331 (+ 232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:40.699993 (+ 1662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:40.701092 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:40.701686 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:40.701809 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:40.702593 (+ 784us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:47:40.702598 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:47:40.702598 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 03:47:40.702793 (+ 195us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:47:40.702831 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 03:47:40.702846 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:47:40.702911 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0806 03:47:40.703029 (+ 118us) server_negotiation.cc:299] Negotiation successful
0806 03:47:40.703108 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":164,"thread_start_us":101,"threads_started":1}
I20260806 03:47:40.714268 25632 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 03:47:40.714983 25632 catalog_manager.cc:1504] Loading token signing keys...
I20260806 03:47:40.721904 25632 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cbed6c26d33945a68a23738f85753b63: Generated new TSK 0
I20260806 03:47:40.722085 25632 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 03:47:40.727022 12279 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskToVWRe/build/release/bin/kudu
/tmp/dist-test-taskToVWRe/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785988004637995-12279-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785988004637995-12279-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785988004637995-12279-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785988004637995-12279-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.253.193:0
--local_ip_for_outbound_sockets=127.11.253.193
--webserver_interface=127.11.253.193
--webserver_port=0
--tserver_master_addrs=127.11.253.254:43437
--builtin_ntp_servers=127.11.253.212:37559
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260806 03:47:40.826936 25650 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:47:40.827179 25650 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:47:40.827227 25650 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:47:40.828418 25650 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:47:40.828485 25650 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:47:40.828526 25650 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.253.193
I20260806 03:47:40.830036 25650 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.253.212:37559
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785988004637995-12279-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785988004637995-12279-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.253.193:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785988004637995-12279-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.253.193
--webserver_port=0
--tserver_master_addrs=127.11.253.254:43437
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25650
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.253.193
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785988004637995-12279-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:47:40.830323 25650 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:47:40.830652 25650 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:47:40.836733 25660 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:47:40.836776 25657 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:47:40.837042 25650 server_base.cc:1034] running on GCE node
W20260806 03:47:40.837117 25656 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:47:40.837316 25650 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:47:40.837590 25650 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:47:40.838747 25650 hybrid_clock.cc:648] HybridClock initialized: now 1785988060838733 us; error 33 us; skew 500 ppm
I20260806 03:47:40.840173 25650 webserver.cc:460] Webserver started at http://127.11.253.193:35333/ using document root <none> and password file <none>
I20260806 03:47:40.840472 25650 fs_manager.cc:359] Metadata directory not provided
I20260806 03:47:40.840533 25650 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:47:40.840685 25650 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:47:40.842162 25650 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785988004637995-12279-0/minicluster-data/ts-0/data/instance:
uuid: "a0e90790a5654abd8c7a553c7bfaeef9"
format_stamp: "Formatted at 2026-08-06 03:47:40 on dist-test-slave-6ntk"
server_key: "e23feb75123c7e9631f12cfe22c66dd1"
server_key_iv: "98281fcdf7ec9cb5be87387920426671"
server_key_version: "encryptionkey@0"
I20260806 03:47:40.842564 25650 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785988004637995-12279-0/minicluster-data/ts-0/wal/instance:
uuid: "a0e90790a5654abd8c7a553c7bfaeef9"
format_stamp: "Formatted at 2026-08-06 03:47:40 on dist-test-slave-6ntk"
server_key: "e23feb75123c7e9631f12cfe22c66dd1"
server_key_iv: "98281fcdf7ec9cb5be87387920426671"
server_key_version: "encryptionkey@0"
I20260806 03:47:40.844326 25650 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260806 03:47:40.845189 25668 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:47:40.845383 25650 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 03:47:40.845481 25650 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785988004637995-12279-0/minicluster-data/ts-0/data,/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785988004637995-12279-0/minicluster-data/ts-0/wal
uuid: "a0e90790a5654abd8c7a553c7bfaeef9"
format_stamp: "Formatted at 2026-08-06 03:47:40 on dist-test-slave-6ntk"
server_key: "e23feb75123c7e9631f12cfe22c66dd1"
server_key_iv: "98281fcdf7ec9cb5be87387920426671"
server_key_version: "encryptionkey@0"
I20260806 03:47:40.845553 25650 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785988004637995-12279-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785988004637995-12279-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785988004637995-12279-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:47:40.876340 25650 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:47:40.876678 25650 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:47:40.876806 25650 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:47:40.877027 25650 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 03:47:40.877475 25650 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 03:47:40.877518 25650 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 03:47:40.877576 25650 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 03:47:40.877607 25650 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 03:47:40.884380 25650 rpc_server.cc:307] RPC server started. Bound to: 127.11.253.193:39341
I20260806 03:47:40.884440 25820 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.253.193:39341 every 8 connection(s)
I20260806 03:47:40.884810 25650 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785988004637995-12279-0/minicluster-data/ts-0/data/info.pb
I20260806 03:47:40.890559 25625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:40.885566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.253.193:39867 (local address 127.11.253.254:43437)
0806 03:47:40.885617 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0806 03:47:40.885619 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:47:40.885949 (+ 330us) server_negotiation.cc:407] Connection header received
0806 03:47:40.886529 (+ 580us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:47:40.886530 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:47:40.886543 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:47:40.886607 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:47:40.887393 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:40.887889 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:40.888972 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:40.889157 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:40.889209 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:40.890201 (+ 992us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:47:40.890203 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:47:40.890203 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 03:47:40.890208 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:47:40.890223 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 03:47:40.890227 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:47:40.890317 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0806 03:47:40.890458 (+ 141us) server_negotiation.cc:299] Negotiation successful
0806 03:47:40.890488 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260806 03:47:40.890698 25823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:40.885529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:43437 (local address 127.11.253.193:39867)
0806 03:47:40.885757 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0806 03:47:40.885804 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0806 03:47:40.886437 (+ 633us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:47:40.886685 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:47:40.886689 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:47:40.886726 (+ 37us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:47:40.887299 (+ 573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:40.887305 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:40.887988 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:40.887988 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:40.888886 (+ 898us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:40.888889 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:40.889246 (+ 357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:40.889247 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:40.889377 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:40.890018 (+ 641us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 03:47:40.890019 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:47:40.890061 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 03:47:40.890086 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:47:40.890328 (+ 242us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:47:40.890334 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:47:40.890337 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 03:47:40.890396 (+ 59us) client_negotiation.cc:240] Negotiation successful
0806 03:47:40.890495 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":102,"threads_started":1}
I20260806 03:47:40.891287 25821 heartbeater.cc:347] Connected to a master server at 127.11.253.254:43437
I20260806 03:47:40.891395 25821 heartbeater.cc:464] Registering TS with master...
I20260806 03:47:40.891618 25821 heartbeater.cc:511] Master 127.11.253.254:43437 requested a full tablet report, sending...
I20260806 03:47:40.892211 25573 ts_manager.cc:194] Registered new tserver with Master: a0e90790a5654abd8c7a553c7bfaeef9 (127.11.253.193:39341)
I20260806 03:47:40.892812 25573 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.11.253.193:39867
I20260806 03:47:40.893630 12279 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskToVWRe/build/release/bin/kudu as pid 25650
I20260806 03:47:40.893700 12279 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785988004637995-12279-0/minicluster-data/ts-0/wal/instance
I20260806 03:47:40.893850 12279 external_mini_cluster.cc:1428] Setting key c815c15f381654bc1bdb06d408ec47fb
I20260806 03:47:40.894894 12279 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskToVWRe/build/release/bin/kudu
/tmp/dist-test-taskToVWRe/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785988004637995-12279-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785988004637995-12279-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785988004637995-12279-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785988004637995-12279-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.253.194:0
--local_ip_for_outbound_sockets=127.11.253.194
--webserver_interface=127.11.253.194
--webserver_port=0
--tserver_master_addrs=127.11.253.254:43437
--builtin_ntp_servers=127.11.253.212:37559
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260806 03:47:40.982254 25826 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:47:40.982475 25826 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:47:40.982497 25826 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:47:40.983680 25826 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:47:40.983721 25826 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:47:40.983745 25826 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.253.194
I20260806 03:47:40.985162 25826 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.253.212:37559
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785988004637995-12279-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785988004637995-12279-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.253.194:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785988004637995-12279-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.253.194
--webserver_port=0
--tserver_master_addrs=127.11.253.254:43437
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25826
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.253.194
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785988004637995-12279-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:47:40.985412 25826 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:47:40.985721 25826 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:47:40.990788 25832 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:47:40.990887 25831 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:47:40.990788 25835 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:47:40.991227 25826 server_base.cc:1034] running on GCE node
I20260806 03:47:40.991451 25826 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:47:40.991675 25826 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:47:40.992835 25826 hybrid_clock.cc:648] HybridClock initialized: now 1785988060992830 us; error 32 us; skew 500 ppm
I20260806 03:47:40.994163 25826 webserver.cc:460] Webserver started at http://127.11.253.194:35555/ using document root <none> and password file <none>
I20260806 03:47:40.994421 25826 fs_manager.cc:359] Metadata directory not provided
I20260806 03:47:40.994467 25826 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:47:40.994558 25826 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:47:40.995571 25826 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785988004637995-12279-0/minicluster-data/ts-1/data/instance:
uuid: "aeb4cb5fa4db41ceab1bf21de8759e8d"
format_stamp: "Formatted at 2026-08-06 03:47:40 on dist-test-slave-6ntk"
server_key: "db91730a0676e917282b73d5228e62b4"
server_key_iv: "be5590712c61e034891b8337fb72968e"
server_key_version: "encryptionkey@0"
I20260806 03:47:40.995869 25826 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785988004637995-12279-0/minicluster-data/ts-1/wal/instance:
uuid: "aeb4cb5fa4db41ceab1bf21de8759e8d"
format_stamp: "Formatted at 2026-08-06 03:47:40 on dist-test-slave-6ntk"
server_key: "db91730a0676e917282b73d5228e62b4"
server_key_iv: "be5590712c61e034891b8337fb72968e"
server_key_version: "encryptionkey@0"
I20260806 03:47:40.997068 25826 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 03:47:40.997742 25841 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:47:40.997952 25826 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 03:47:40.998041 25826 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785988004637995-12279-0/minicluster-data/ts-1/data,/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785988004637995-12279-0/minicluster-data/ts-1/wal
uuid: "aeb4cb5fa4db41ceab1bf21de8759e8d"
format_stamp: "Formatted at 2026-08-06 03:47:40 on dist-test-slave-6ntk"
server_key: "db91730a0676e917282b73d5228e62b4"
server_key_iv: "be5590712c61e034891b8337fb72968e"
server_key_version: "encryptionkey@0"
I20260806 03:47:40.998131 25826 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785988004637995-12279-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785988004637995-12279-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785988004637995-12279-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:47:41.019347 25826 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:47:41.019701 25826 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:47:41.019863 25826 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:47:41.020139 25826 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 03:47:41.020509 25826 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 03:47:41.020543 25826 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 03:47:41.020566 25826 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 03:47:41.020601 25826 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 03:47:41.026522 25826 rpc_server.cc:307] RPC server started. Bound to: 127.11.253.194:45555
I20260806 03:47:41.026605 26004 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.253.194:45555 every 8 connection(s)
I20260806 03:47:41.027001 25826 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785988004637995-12279-0/minicluster-data/ts-1/data/info.pb
I20260806 03:47:41.030977 12279 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskToVWRe/build/release/bin/kudu as pid 25826
I20260806 03:47:41.031112 12279 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785988004637995-12279-0/minicluster-data/ts-1/wal/instance
I20260806 03:47:41.031371 12279 external_mini_cluster.cc:1428] Setting key f1bb59202c5cc33d020159ff08a4489e
I20260806 03:47:41.032909 12279 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskToVWRe/build/release/bin/kudu
/tmp/dist-test-taskToVWRe/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785988004637995-12279-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785988004637995-12279-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785988004637995-12279-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785988004637995-12279-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.253.195:0
--local_ip_for_outbound_sockets=127.11.253.195
--webserver_interface=127.11.253.195
--webserver_port=0
--tserver_master_addrs=127.11.253.254:43437
--builtin_ntp_servers=127.11.253.212:37559
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260806 03:47:41.033130 25625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:41.027886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.253.194:53581 (local address 127.11.253.254:43437)
0806 03:47:41.027982 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0806 03:47:41.027984 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:47:41.028229 (+ 245us) server_negotiation.cc:407] Connection header received
0806 03:47:41.028732 (+ 503us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:47:41.028733 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:47:41.028744 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:47:41.028802 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:47:41.029531 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:41.030162 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:41.031297 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:41.031651 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:41.031736 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:41.032765 (+ 1029us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:47:41.032766 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:47:41.032766 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 03:47:41.032771 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:47:41.032786 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 03:47:41.032790 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:47:41.032839 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0806 03:47:41.032958 (+ 119us) server_negotiation.cc:299] Negotiation successful
0806 03:47:41.033010 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260806 03:47:41.033329 26007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:41.027848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:43437 (local address 127.11.253.194:53581)
0806 03:47:41.028092 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0806 03:47:41.028129 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0806 03:47:41.028639 (+ 510us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:47:41.028856 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:47:41.028860 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:47:41.028868 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:47:41.029450 (+ 582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:41.029455 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:41.030260 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:41.030262 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:41.031075 (+ 813us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:41.031077 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:41.031769 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:41.031770 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:41.031885 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:41.032590 (+ 705us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 03:47:41.032591 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:47:41.032632 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 03:47:41.032661 (+ 29us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:47:41.032861 (+ 200us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:47:41.032868 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:47:41.032871 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 03:47:41.032952 (+ 81us) client_negotiation.cc:240] Negotiation successful
0806 03:47:41.033075 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":131,"threads_started":1}
I20260806 03:47:41.033798 26005 heartbeater.cc:347] Connected to a master server at 127.11.253.254:43437
I20260806 03:47:41.033941 26005 heartbeater.cc:464] Registering TS with master...
I20260806 03:47:41.034143 26005 heartbeater.cc:511] Master 127.11.253.254:43437 requested a full tablet report, sending...
I20260806 03:47:41.034581 25573 ts_manager.cc:194] Registered new tserver with Master: aeb4cb5fa4db41ceab1bf21de8759e8d (127.11.253.194:45555)
I20260806 03:47:41.035162 25573 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.11.253.194:53581
W20260806 03:47:41.123723 26008 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:47:41.123946 26008 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:47:41.123970 26008 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:47:41.125078 26008 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:47:41.125113 26008 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:47:41.125136 26008 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.253.195
I20260806 03:47:41.126931 26008 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.253.212:37559
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785988004637995-12279-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785988004637995-12279-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.253.195:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785988004637995-12279-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.253.195
--webserver_port=0
--tserver_master_addrs=127.11.253.254:43437
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26008
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.253.195
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785988004637995-12279-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:47:41.127254 26008 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:47:41.127645 26008 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:47:41.133160 26016 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:47:41.133216 26017 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:47:41.133229 26008 server_base.cc:1034] running on GCE node
W20260806 03:47:41.133383 26020 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:47:41.133626 26008 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:47:41.133920 26008 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:47:41.135079 26008 hybrid_clock.cc:648] HybridClock initialized: now 1785988061135067 us; error 35 us; skew 500 ppm
I20260806 03:47:41.136502 26008 webserver.cc:460] Webserver started at http://127.11.253.195:41049/ using document root <none> and password file <none>
I20260806 03:47:41.136787 26008 fs_manager.cc:359] Metadata directory not provided
I20260806 03:47:41.136853 26008 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:47:41.137010 26008 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:47:41.138253 26008 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785988004637995-12279-0/minicluster-data/ts-2/data/instance:
uuid: "5122d6fcf81047a483786db488f5c77b"
format_stamp: "Formatted at 2026-08-06 03:47:41 on dist-test-slave-6ntk"
server_key: "02fd26a47a39eaa2a295875e6d76ce01"
server_key_iv: "214e23f8613f0ef89a045580517c9a19"
server_key_version: "encryptionkey@0"
I20260806 03:47:41.138638 26008 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785988004637995-12279-0/minicluster-data/ts-2/wal/instance:
uuid: "5122d6fcf81047a483786db488f5c77b"
format_stamp: "Formatted at 2026-08-06 03:47:41 on dist-test-slave-6ntk"
server_key: "02fd26a47a39eaa2a295875e6d76ce01"
server_key_iv: "214e23f8613f0ef89a045580517c9a19"
server_key_version: "encryptionkey@0"
I20260806 03:47:41.140091 26008 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260806 03:47:41.140913 26027 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:47:41.141124 26008 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 03:47:41.141228 26008 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785988004637995-12279-0/minicluster-data/ts-2/data,/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785988004637995-12279-0/minicluster-data/ts-2/wal
uuid: "5122d6fcf81047a483786db488f5c77b"
format_stamp: "Formatted at 2026-08-06 03:47:41 on dist-test-slave-6ntk"
server_key: "02fd26a47a39eaa2a295875e6d76ce01"
server_key_iv: "214e23f8613f0ef89a045580517c9a19"
server_key_version: "encryptionkey@0"
I20260806 03:47:41.141348 26008 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785988004637995-12279-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785988004637995-12279-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785988004637995-12279-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:47:41.152901 26008 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:47:41.153208 26008 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:47:41.153364 26008 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:47:41.153594 26008 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 03:47:41.153992 26008 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 03:47:41.154031 26008 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 03:47:41.154054 26008 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 03:47:41.154066 26008 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 03:47:41.160315 26008 rpc_server.cc:307] RPC server started. Bound to: 127.11.253.195:39857
I20260806 03:47:41.160385 26183 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.253.195:39857 every 8 connection(s)
I20260806 03:47:41.160750 26008 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785988004637995-12279-0/minicluster-data/ts-2/data/info.pb
I20260806 03:47:41.166265 25625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:41.161528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.253.195:55405 (local address 127.11.253.254:43437)
0806 03:47:41.161574 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0806 03:47:41.161576 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:47:41.161805 (+ 229us) server_negotiation.cc:407] Connection header received
0806 03:47:41.162412 (+ 607us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:47:41.162414 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:47:41.162428 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:47:41.162480 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:47:41.163189 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:41.163696 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:41.164701 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:41.164912 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:41.164992 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:41.165959 (+ 967us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:47:41.165960 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:47:41.165960 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 03:47:41.165966 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:47:41.165980 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 03:47:41.165984 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:47:41.166048 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0806 03:47:41.166150 (+ 102us) server_negotiation.cc:299] Negotiation successful
0806 03:47:41.166182 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 03:47:41.166481 26186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:41.161490 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:43437 (local address 127.11.253.195:55405)
0806 03:47:41.161678 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0806 03:47:41.161712 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0806 03:47:41.162274 (+ 562us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:47:41.162562 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:47:41.162566 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:47:41.162578 (+ 12us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:47:41.163090 (+ 512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:41.163094 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:41.163799 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:41.163800 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:41.164605 (+ 805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:41.164607 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:41.165003 (+ 396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:41.165004 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:41.165150 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:41.165685 (+ 535us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 03:47:41.165685 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:47:41.165714 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 03:47:41.165737 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:47:41.166050 (+ 313us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:47:41.166057 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:47:41.166060 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 03:47:41.166151 (+ 91us) client_negotiation.cc:240] Negotiation successful
0806 03:47:41.166256 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":70,"threads_started":1}
I20260806 03:47:41.166985 26184 heartbeater.cc:347] Connected to a master server at 127.11.253.254:43437
I20260806 03:47:41.167091 26184 heartbeater.cc:464] Registering TS with master...
I20260806 03:47:41.167275 26184 heartbeater.cc:511] Master 127.11.253.254:43437 requested a full tablet report, sending...
I20260806 03:47:41.167657 25573 ts_manager.cc:194] Registered new tserver with Master: 5122d6fcf81047a483786db488f5c77b (127.11.253.195:39857)
I20260806 03:47:41.167994 25573 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.11.253.195:55405
I20260806 03:47:41.169859 12279 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskToVWRe/build/release/bin/kudu as pid 26008
I20260806 03:47:41.170018 12279 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785988004637995-12279-0/minicluster-data/ts-2/wal/instance
I20260806 03:47:41.170250 12279 external_mini_cluster.cc:1428] Setting key 28d70c8e5013c08888bfad74475ce42b
I20260806 03:47:41.171303 12279 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 03:47:41.173241 12279 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskToVWRe/build/release/bin/kudu with pid 25650
W20260806 03:47:41.173283 25625 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 03:47:41.172737 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51132 (local address 127.11.253.254:43437)
0806 03:47:41.172784 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0806 03:47:41.172787 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 03:47:41.172915 (+ 128us) server_negotiation.cc:407] Connection header received
0806 03:47:41.172943 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:47:41.172944 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:47:41.172956 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:47:41.173002 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:47:41.173182 (+ 180us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51132: BlockingRecv error: recv got EOF from 127.0.0.1:51132 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
I20260806 03:47:41.184580 12279 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskToVWRe/build/release/bin/kudu with pid 25826
I20260806 03:47:41.194010 12279 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskToVWRe/build/release/bin/kudu with pid 26008
I20260806 03:47:41.202891 12279 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskToVWRe/build/release/bin/kudu with pid 25520
2026-08-06T03:47:41Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (685 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-06T03:47:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T03:47:41Z Disabled control of system clock
I20260806 03:47:41.237258 12279 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskToVWRe/build/release/bin/kudu
/tmp/dist-test-taskToVWRe/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785988004637995-12279-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785988004637995-12279-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785988004637995-12279-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785988004637995-12279-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.253.254:44363
--webserver_interface=127.11.253.254
--webserver_port=0
--builtin_ntp_servers=127.11.253.212:42253
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.253.254:44363
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260806 03:47:41.320884 26200 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:47:41.321100 26200 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:47:41.321123 26200 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:47:41.322336 26200 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 03:47:41.322381 26200 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:47:41.322393 26200 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:47:41.322405 26200 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 03:47:41.322418 26200 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 03:47:41.323695 26200 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.253.212:42253
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785988004637995-12279-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785988004637995-12279-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.253.254:44363
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.253.254:44363
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785988004637995-12279-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.253.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26200
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785988004637995-12279-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:47:41.323909 26200 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:47:41.324163 26200 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:47:41.329298 26206 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:47:41.329310 26209 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:47:41.329310 26207 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:47:41.329689 26200 server_base.cc:1034] running on GCE node
I20260806 03:47:41.330173 26200 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:47:41.330502 26200 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:47:41.331676 26200 hybrid_clock.cc:648] HybridClock initialized: now 1785988061331657 us; error 35 us; skew 500 ppm
I20260806 03:47:41.333132 26200 webserver.cc:460] Webserver started at http://127.11.253.254:33467/ using document root <none> and password file <none>
I20260806 03:47:41.333408 26200 fs_manager.cc:359] Metadata directory not provided
I20260806 03:47:41.333483 26200 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:47:41.333643 26200 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:47:41.334816 26200 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785988004637995-12279-0/minicluster-data/master-0/data/instance:
uuid: "3411516647c043cf911bc2ef19d41c59"
format_stamp: "Formatted at 2026-08-06 03:47:41 on dist-test-slave-6ntk"
server_key: "591f37f136b56efbfa4dae5fcf1183bc"
server_key_iv: "a05f9bb65155502a3977f2f6e22b7729"
server_key_version: "encryptionkey@0"
I20260806 03:47:41.335193 26200 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785988004637995-12279-0/minicluster-data/master-0/wal/instance:
uuid: "3411516647c043cf911bc2ef19d41c59"
format_stamp: "Formatted at 2026-08-06 03:47:41 on dist-test-slave-6ntk"
server_key: "591f37f136b56efbfa4dae5fcf1183bc"
server_key_iv: "a05f9bb65155502a3977f2f6e22b7729"
server_key_version: "encryptionkey@0"
I20260806 03:47:41.336550 26200 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 03:47:41.337285 26217 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:47:41.337510 26200 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 03:47:41.337623 26200 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785988004637995-12279-0/minicluster-data/master-0/data,/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785988004637995-12279-0/minicluster-data/master-0/wal
uuid: "3411516647c043cf911bc2ef19d41c59"
format_stamp: "Formatted at 2026-08-06 03:47:41 on dist-test-slave-6ntk"
server_key: "591f37f136b56efbfa4dae5fcf1183bc"
server_key_iv: "a05f9bb65155502a3977f2f6e22b7729"
server_key_version: "encryptionkey@0"
I20260806 03:47:41.337700 26200 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785988004637995-12279-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785988004637995-12279-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785988004637995-12279-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:47:41.341758 26200 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:47:41.342083 26200 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:47:41.342222 26200 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:47:41.347441 26200 rpc_server.cc:307] RPC server started. Bound to: 127.11.253.254:44363
I20260806 03:47:41.347580 26289 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.253.254:44363 every 8 connection(s)
I20260806 03:47:41.347923 26200 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785988004637995-12279-0/minicluster-data/master-0/data/info.pb
I20260806 03:47:41.348798 26290 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:47:41.353724 26290 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3411516647c043cf911bc2ef19d41c59: Bootstrap starting.
I20260806 03:47:41.353924 12279 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskToVWRe/build/release/bin/kudu as pid 26200
I20260806 03:47:41.354025 12279 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785988004637995-12279-0/minicluster-data/master-0/wal/instance
I20260806 03:47:41.354231 12279 external_mini_cluster.cc:1428] Setting key 73351ddb1c9f44d1d0678475e53ba996
I20260806 03:47:41.354697 26290 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3411516647c043cf911bc2ef19d41c59: Neither blocks nor log segments found. Creating new log.
I20260806 03:47:41.355093 26290 log.cc:824] T 00000000000000000000000000000000 P 3411516647c043cf911bc2ef19d41c59: Log is configured to *not* fsync() on all Append() calls
I20260806 03:47:41.356045 26290 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3411516647c043cf911bc2ef19d41c59: No bootstrap required, opened a new log
I20260806 03:47:41.357496 26293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:41.354958 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50006 (local address 127.11.253.254:44363)
0806 03:47:41.355188 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0806 03:47:41.355197 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 03:47:41.355232 (+ 35us) server_negotiation.cc:407] Connection header received
0806 03:47:41.355826 (+ 594us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:47:41.355834 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:47:41.355877 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:47:41.356007 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:47:41.356880 (+ 873us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:47:41.356884 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:47:41.356885 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 03:47:41.356890 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:47:41.356939 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 03:47:41.356955 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:47:41.357054 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0806 03:47:41.357211 (+ 157us) server_negotiation.cc:299] Negotiation successful
0806 03:47:41.357276 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":102,"threads_started":1}
I20260806 03:47:41.358439 26290 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3411516647c043cf911bc2ef19d41c59 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3411516647c043cf911bc2ef19d41c59" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 44363 } }
I20260806 03:47:41.358616 26290 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3411516647c043cf911bc2ef19d41c59 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:47:41.358654 26290 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3411516647c043cf911bc2ef19d41c59 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3411516647c043cf911bc2ef19d41c59, State: Initialized, Role: FOLLOWER
I20260806 03:47:41.358815 26290 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3411516647c043cf911bc2ef19d41c59 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3411516647c043cf911bc2ef19d41c59" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 44363 } }
I20260806 03:47:41.358928 26290 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3411516647c043cf911bc2ef19d41c59 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 03:47:41.358982 26290 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3411516647c043cf911bc2ef19d41c59 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 03:47:41.359041 26290 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3411516647c043cf911bc2ef19d41c59 [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:47:41.360080 26290 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3411516647c043cf911bc2ef19d41c59 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3411516647c043cf911bc2ef19d41c59" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 44363 } }
I20260806 03:47:41.360209 26290 leader_election.cc:302] T 00000000000000000000000000000000 P 3411516647c043cf911bc2ef19d41c59 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3411516647c043cf911bc2ef19d41c59; no voters:
I20260806 03:47:41.360399 26290 leader_election.cc:288] T 00000000000000000000000000000000 P 3411516647c043cf911bc2ef19d41c59 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 03:47:41.360438 26296 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3411516647c043cf911bc2ef19d41c59 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 03:47:41.360558 26296 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3411516647c043cf911bc2ef19d41c59 [term 1 LEADER]: Becoming Leader. State: Replica: 3411516647c043cf911bc2ef19d41c59, State: Running, Role: LEADER
I20260806 03:47:41.360659 26296 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3411516647c043cf911bc2ef19d41c59 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3411516647c043cf911bc2ef19d41c59" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 44363 } }
I20260806 03:47:41.360687 26290 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3411516647c043cf911bc2ef19d41c59 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 03:47:41.360988 26296 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3411516647c043cf911bc2ef19d41c59 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3411516647c043cf911bc2ef19d41c59. Latest consensus state: current_term: 1 leader_uuid: "3411516647c043cf911bc2ef19d41c59" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3411516647c043cf911bc2ef19d41c59" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 44363 } } }
I20260806 03:47:41.361078 26296 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3411516647c043cf911bc2ef19d41c59 [sys.catalog]: This master's current role is: LEADER
I20260806 03:47:41.360998 26300 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3411516647c043cf911bc2ef19d41c59 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3411516647c043cf911bc2ef19d41c59" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3411516647c043cf911bc2ef19d41c59" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 44363 } } }
I20260806 03:47:41.361254 26300 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3411516647c043cf911bc2ef19d41c59 [sys.catalog]: This master's current role is: LEADER
I20260806 03:47:41.362051 26303 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 03:47:41.362558 26303 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 03:47:41.364388 26303 catalog_manager.cc:1347] Generated new cluster ID: 201bc12cdaf54b07ad5fcd068b7273bc
I20260806 03:47:41.364444 26303 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 03:47:41.370599 26303 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 03:47:41.370765 26303 catalog_manager.cc:1504] Loading token signing keys...
I20260806 03:47:41.374909 26303 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3411516647c043cf911bc2ef19d41c59: Generated new TSK 0
I20260806 03:47:41.375093 26303 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 03:47:41.378237 12279 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskToVWRe/build/release/bin/kudu
/tmp/dist-test-taskToVWRe/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785988004637995-12279-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785988004637995-12279-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785988004637995-12279-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785988004637995-12279-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.253.193:0
--local_ip_for_outbound_sockets=127.11.253.193
--webserver_interface=127.11.253.193
--webserver_port=0
--tserver_master_addrs=127.11.253.254:44363
--builtin_ntp_servers=127.11.253.212:42253
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260806 03:47:41.477671 26322 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:47:41.478055 26322 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:47:41.478107 26322 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:47:41.479460 26322 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:47:41.479537 26322 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:47:41.479591 26322 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.253.193
I20260806 03:47:41.481150 26322 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.253.212:42253
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785988004637995-12279-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785988004637995-12279-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.253.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785988004637995-12279-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.253.193
--webserver_port=0
--tserver_master_addrs=127.11.253.254:44363
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26322
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.253.193
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785988004637995-12279-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:47:41.481469 26322 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:47:41.481851 26322 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:47:41.488188 26332 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:47:41.488164 26336 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:47:41.488173 26331 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:47:41.488694 26322 server_base.cc:1034] running on GCE node
I20260806 03:47:41.488937 26322 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:47:41.489215 26322 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:47:41.490396 26322 hybrid_clock.cc:648] HybridClock initialized: now 1785988061490383 us; error 31 us; skew 500 ppm
I20260806 03:47:41.491912 26322 webserver.cc:460] Webserver started at http://127.11.253.193:39801/ using document root <none> and password file <none>
I20260806 03:47:41.492195 26322 fs_manager.cc:359] Metadata directory not provided
I20260806 03:47:41.492266 26322 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:47:41.492427 26322 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:47:41.493901 26322 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785988004637995-12279-0/minicluster-data/ts-0/data/instance:
uuid: "cf25f2f60a474295bdd4dd2248681a4f"
format_stamp: "Formatted at 2026-08-06 03:47:41 on dist-test-slave-6ntk"
server_key: "05ee5643043d40558df8dc7c55e1ad63"
server_key_iv: "a0a30a1f5e08ddb866a8e5c6e1f9bda3"
server_key_version: "encryptionkey@0"
I20260806 03:47:41.494331 26322 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785988004637995-12279-0/minicluster-data/ts-0/wal/instance:
uuid: "cf25f2f60a474295bdd4dd2248681a4f"
format_stamp: "Formatted at 2026-08-06 03:47:41 on dist-test-slave-6ntk"
server_key: "05ee5643043d40558df8dc7c55e1ad63"
server_key_iv: "a0a30a1f5e08ddb866a8e5c6e1f9bda3"
server_key_version: "encryptionkey@0"
I20260806 03:47:41.496091 26322 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260806 03:47:41.497031 26343 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:47:41.497228 26322 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 03:47:41.497323 26322 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785988004637995-12279-0/minicluster-data/ts-0/data,/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785988004637995-12279-0/minicluster-data/ts-0/wal
uuid: "cf25f2f60a474295bdd4dd2248681a4f"
format_stamp: "Formatted at 2026-08-06 03:47:41 on dist-test-slave-6ntk"
server_key: "05ee5643043d40558df8dc7c55e1ad63"
server_key_iv: "a0a30a1f5e08ddb866a8e5c6e1f9bda3"
server_key_version: "encryptionkey@0"
I20260806 03:47:41.497418 26322 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785988004637995-12279-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785988004637995-12279-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785988004637995-12279-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:47:41.500689 26322 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:47:41.500974 26322 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:47:41.501121 26322 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:47:41.501329 26322 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 03:47:41.501655 26322 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 03:47:41.501711 26322 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 03:47:41.501751 26322 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 03:47:41.501780 26322 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 03:47:41.507454 26322 rpc_server.cc:307] RPC server started. Bound to: 127.11.253.193:35695
I20260806 03:47:41.507571 26495 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.253.193:35695 every 8 connection(s)
I20260806 03:47:41.507887 26322 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785988004637995-12279-0/minicluster-data/ts-0/data/info.pb
I20260806 03:47:41.511701 26293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:41.508769 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.253.193:42967 (local address 127.11.253.254:44363)
0806 03:47:41.508823 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0806 03:47:41.508825 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:47:41.509092 (+ 267us) server_negotiation.cc:407] Connection header received
0806 03:47:41.509779 (+ 687us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:47:41.509780 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:47:41.509787 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:47:41.509855 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:47:41.511370 (+ 1515us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:47:41.511372 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:47:41.511372 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 03:47:41.511377 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:47:41.511394 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 03:47:41.511399 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:47:41.511477 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0806 03:47:41.511606 (+ 129us) server_negotiation.cc:299] Negotiation successful
0806 03:47:41.511621 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260806 03:47:41.511963 26499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:41.508744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:44363 (local address 127.11.253.193:42967)
0806 03:47:41.508936 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0806 03:47:41.508981 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 03:47:41.509661 (+ 680us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:47:41.509968 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:47:41.509973 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:47:41.509983 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:47:41.511143 (+ 1160us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 03:47:41.511143 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:47:41.511183 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 03:47:41.511220 (+ 37us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:47:41.511495 (+ 275us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:47:41.511504 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:47:41.511508 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 03:47:41.511608 (+ 100us) client_negotiation.cc:240] Negotiation successful
0806 03:47:41.511681 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":89,"threads_started":1}
I20260806 03:47:41.512723 26497 heartbeater.cc:347] Connected to a master server at 127.11.253.254:44363
I20260806 03:47:41.512837 26497 heartbeater.cc:464] Registering TS with master...
I20260806 03:47:41.513021 26497 heartbeater.cc:511] Master 127.11.253.254:44363 requested a full tablet report, sending...
I20260806 03:47:41.513581 26239 ts_manager.cc:194] Registered new tserver with Master: cf25f2f60a474295bdd4dd2248681a4f (127.11.253.193:35695)
I20260806 03:47:41.515013 12279 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskToVWRe/build/release/bin/kudu as pid 26322
I20260806 03:47:41.515097 12279 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785988004637995-12279-0/minicluster-data/ts-0/wal/instance
I20260806 03:47:41.515321 12279 external_mini_cluster.cc:1428] Setting key 2fc47c692e176a7fa7d2f6567fcb8749
I20260806 03:47:41.516268 12279 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskToVWRe/build/release/bin/kudu
/tmp/dist-test-taskToVWRe/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785988004637995-12279-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785988004637995-12279-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785988004637995-12279-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785988004637995-12279-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.253.194:0
--local_ip_for_outbound_sockets=127.11.253.194
--webserver_interface=127.11.253.194
--webserver_port=0
--tserver_master_addrs=127.11.253.254:44363
--builtin_ntp_servers=127.11.253.212:42253
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260806 03:47:41.601310 26500 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:47:41.601541 26500 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:47:41.601563 26500 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:47:41.602756 26500 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:47:41.602797 26500 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:47:41.602823 26500 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.253.194
I20260806 03:47:41.604187 26500 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.253.212:42253
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785988004637995-12279-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785988004637995-12279-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.253.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785988004637995-12279-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.253.194
--webserver_port=0
--tserver_master_addrs=127.11.253.254:44363
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26500
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.253.194
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785988004637995-12279-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:47:41.604460 26500 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:47:41.604755 26500 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 03:47:41.610091 26500 server_base.cc:1034] running on GCE node
W20260806 03:47:41.610056 26508 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:47:41.610054 26507 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:47:41.610036 26510 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:47:41.610464 26500 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:47:41.610745 26500 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:47:41.611930 26500 hybrid_clock.cc:648] HybridClock initialized: now 1785988061611931 us; error 43 us; skew 500 ppm
I20260806 03:47:41.613176 26500 webserver.cc:460] Webserver started at http://127.11.253.194:40769/ using document root <none> and password file <none>
I20260806 03:47:41.613427 26500 fs_manager.cc:359] Metadata directory not provided
I20260806 03:47:41.613499 26500 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:47:41.613659 26500 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:47:41.614789 26500 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785988004637995-12279-0/minicluster-data/ts-1/data/instance:
uuid: "386b728125184a03ba966bdf214fe35d"
format_stamp: "Formatted at 2026-08-06 03:47:41 on dist-test-slave-6ntk"
server_key: "50075d9330ef93a75670d6940dca6716"
server_key_iv: "e6ba82dc63583d41c4f0b472de6cfde7"
server_key_version: "encryptionkey@0"
I20260806 03:47:41.615170 26500 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785988004637995-12279-0/minicluster-data/ts-1/wal/instance:
uuid: "386b728125184a03ba966bdf214fe35d"
format_stamp: "Formatted at 2026-08-06 03:47:41 on dist-test-slave-6ntk"
server_key: "50075d9330ef93a75670d6940dca6716"
server_key_iv: "e6ba82dc63583d41c4f0b472de6cfde7"
server_key_version: "encryptionkey@0"
I20260806 03:47:41.616571 26500 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260806 03:47:41.617378 26519 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:47:41.617599 26500 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 03:47:41.617707 26500 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785988004637995-12279-0/minicluster-data/ts-1/data,/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785988004637995-12279-0/minicluster-data/ts-1/wal
uuid: "386b728125184a03ba966bdf214fe35d"
format_stamp: "Formatted at 2026-08-06 03:47:41 on dist-test-slave-6ntk"
server_key: "50075d9330ef93a75670d6940dca6716"
server_key_iv: "e6ba82dc63583d41c4f0b472de6cfde7"
server_key_version: "encryptionkey@0"
I20260806 03:47:41.617797 26500 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785988004637995-12279-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785988004637995-12279-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785988004637995-12279-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:47:41.621167 26500 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:47:41.621449 26500 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:47:41.621577 26500 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:47:41.621779 26500 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 03:47:41.622133 26500 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 03:47:41.622182 26500 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 03:47:41.622215 26500 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 03:47:41.622241 26500 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 03:47:41.628139 26500 rpc_server.cc:307] RPC server started. Bound to: 127.11.253.194:43173
I20260806 03:47:41.628182 26670 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.253.194:43173 every 8 connection(s)
I20260806 03:47:41.628509 26500 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785988004637995-12279-0/minicluster-data/ts-1/data/info.pb
I20260806 03:47:41.631670 26293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:41.629324 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.253.194:38421 (local address 127.11.253.254:44363)
0806 03:47:41.629412 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0806 03:47:41.629414 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:47:41.629649 (+ 235us) server_negotiation.cc:407] Connection header received
0806 03:47:41.630210 (+ 561us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:47:41.630212 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:47:41.630217 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:47:41.630274 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:47:41.631442 (+ 1168us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:47:41.631443 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:47:41.631444 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 03:47:41.631448 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:47:41.631462 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 03:47:41.631465 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:47:41.631513 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0806 03:47:41.631597 (+ 84us) server_negotiation.cc:299] Negotiation successful
0806 03:47:41.631610 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260806 03:47:41.631872 26675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:41.629286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:44363 (local address 127.11.253.194:38421)
0806 03:47:41.629526 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0806 03:47:41.629561 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0806 03:47:41.630120 (+ 559us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:47:41.630370 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:47:41.630373 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:47:41.630382 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:47:41.631299 (+ 917us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 03:47:41.631300 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:47:41.631328 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 03:47:41.631357 (+ 29us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:47:41.631527 (+ 170us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:47:41.631532 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:47:41.631534 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 03:47:41.631594 (+ 60us) client_negotiation.cc:240] Negotiation successful
0806 03:47:41.631645 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":122,"threads_started":1}
I20260806 03:47:41.632166 12279 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskToVWRe/build/release/bin/kudu as pid 26500
I20260806 03:47:41.632293 12279 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785988004637995-12279-0/minicluster-data/ts-1/wal/instance
I20260806 03:47:41.632390 26672 heartbeater.cc:347] Connected to a master server at 127.11.253.254:44363
I20260806 03:47:41.632512 26672 heartbeater.cc:464] Registering TS with master...
I20260806 03:47:41.632512 12279 external_mini_cluster.cc:1428] Setting key 7a2d77b91ac5b98d7c5afcbe27e04d3c
I20260806 03:47:41.632688 26672 heartbeater.cc:511] Master 127.11.253.254:44363 requested a full tablet report, sending...
I20260806 03:47:41.633112 26238 ts_manager.cc:194] Registered new tserver with Master: 386b728125184a03ba966bdf214fe35d (127.11.253.194:43173)
I20260806 03:47:41.633500 12279 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskToVWRe/build/release/bin/kudu
/tmp/dist-test-taskToVWRe/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785988004637995-12279-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785988004637995-12279-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785988004637995-12279-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785988004637995-12279-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.253.195:0
--local_ip_for_outbound_sockets=127.11.253.195
--webserver_interface=127.11.253.195
--webserver_port=0
--tserver_master_addrs=127.11.253.254:44363
--builtin_ntp_servers=127.11.253.212:42253
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260806 03:47:41.716980 26676 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:47:41.717199 26676 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:47:41.717221 26676 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:47:41.718416 26676 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:47:41.718456 26676 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:47:41.718482 26676 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.253.195
I20260806 03:47:41.719823 26676 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.253.212:42253
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785988004637995-12279-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785988004637995-12279-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.253.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785988004637995-12279-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.253.195
--webserver_port=0
--tserver_master_addrs=127.11.253.254:44363
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26676
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.253.195
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785988004637995-12279-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:47:41.720033 26676 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:47:41.720403 26676 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:47:41.725842 26684 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:47:41.725841 26683 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:47:41.725947 26686 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:47:41.726115 26676 server_base.cc:1034] running on GCE node
I20260806 03:47:41.726356 26676 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:47:41.726612 26676 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:47:41.727777 26676 hybrid_clock.cc:648] HybridClock initialized: now 1785988061727759 us; error 48 us; skew 500 ppm
I20260806 03:47:41.729262 26676 webserver.cc:460] Webserver started at http://127.11.253.195:38983/ using document root <none> and password file <none>
I20260806 03:47:41.729538 26676 fs_manager.cc:359] Metadata directory not provided
I20260806 03:47:41.729610 26676 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:47:41.729756 26676 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:47:41.730901 26676 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785988004637995-12279-0/minicluster-data/ts-2/data/instance:
uuid: "1f5b5a7fe00a4706890468fb4ac7289e"
format_stamp: "Formatted at 2026-08-06 03:47:41 on dist-test-slave-6ntk"
server_key: "12670f177c1617735845fdab94930e35"
server_key_iv: "8e6dfcedce0dd926bc3e95604eab98c7"
server_key_version: "encryptionkey@0"
I20260806 03:47:41.731278 26676 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785988004637995-12279-0/minicluster-data/ts-2/wal/instance:
uuid: "1f5b5a7fe00a4706890468fb4ac7289e"
format_stamp: "Formatted at 2026-08-06 03:47:41 on dist-test-slave-6ntk"
server_key: "12670f177c1617735845fdab94930e35"
server_key_iv: "8e6dfcedce0dd926bc3e95604eab98c7"
server_key_version: "encryptionkey@0"
I20260806 03:47:41.732683 26676 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 03:47:41.733430 26695 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:47:41.733615 26676 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 03:47:41.733742 26676 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785988004637995-12279-0/minicluster-data/ts-2/data,/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785988004637995-12279-0/minicluster-data/ts-2/wal
uuid: "1f5b5a7fe00a4706890468fb4ac7289e"
format_stamp: "Formatted at 2026-08-06 03:47:41 on dist-test-slave-6ntk"
server_key: "12670f177c1617735845fdab94930e35"
server_key_iv: "8e6dfcedce0dd926bc3e95604eab98c7"
server_key_version: "encryptionkey@0"
I20260806 03:47:41.733855 26676 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785988004637995-12279-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785988004637995-12279-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785988004637995-12279-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:47:41.737238 26676 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:47:41.737521 26676 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:47:41.737656 26676 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:47:41.737936 26676 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 03:47:41.738288 26676 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 03:47:41.738345 26676 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 03:47:41.738385 26676 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 03:47:41.738412 26676 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 03:47:41.744323 26676 rpc_server.cc:307] RPC server started. Bound to: 127.11.253.195:38305
I20260806 03:47:41.744355 26846 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.253.195:38305 every 8 connection(s)
I20260806 03:47:41.744681 26676 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785988004637995-12279-0/minicluster-data/ts-2/data/info.pb
I20260806 03:47:41.747635 26293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:41.745425 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.253.195:46407 (local address 127.11.253.254:44363)
0806 03:47:41.745478 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0806 03:47:41.745480 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:47:41.745695 (+ 215us) server_negotiation.cc:407] Connection header received
0806 03:47:41.746225 (+ 530us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:47:41.746226 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:47:41.746232 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:47:41.746284 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:47:41.747474 (+ 1190us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:47:41.747475 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:47:41.747475 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 03:47:41.747477 (+ 2us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:47:41.747487 (+ 10us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 03:47:41.747490 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:47:41.747528 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0806 03:47:41.747573 (+ 45us) server_negotiation.cc:299] Negotiation successful
0806 03:47:41.747584 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260806 03:47:41.747828 26853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:41.745415 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:44363 (local address 127.11.253.195:46407)
0806 03:47:41.745596 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0806 03:47:41.745628 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0806 03:47:41.746136 (+ 508us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:47:41.746335 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:47:41.746338 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:47:41.746346 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:47:41.747349 (+ 1003us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 03:47:41.747350 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:47:41.747377 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 03:47:41.747404 (+ 27us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:47:41.747529 (+ 125us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:47:41.747532 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:47:41.747534 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 03:47:41.747580 (+ 46us) client_negotiation.cc:240] Negotiation successful
0806 03:47:41.747624 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":90,"threads_started":1}
I20260806 03:47:41.748327 26847 heartbeater.cc:347] Connected to a master server at 127.11.253.254:44363
I20260806 03:47:41.748425 26847 heartbeater.cc:464] Registering TS with master...
I20260806 03:47:41.748534 26847 heartbeater.cc:511] Master 127.11.253.254:44363 requested a full tablet report, sending...
I20260806 03:47:41.748893 26238 ts_manager.cc:194] Registered new tserver with Master: 1f5b5a7fe00a4706890468fb4ac7289e (127.11.253.195:38305)
I20260806 03:47:41.749166 12279 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskToVWRe/build/release/bin/kudu as pid 26676
I20260806 03:47:41.749248 12279 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785988004637995-12279-0/minicluster-data/ts-2/wal/instance
I20260806 03:47:41.749437 12279 external_mini_cluster.cc:1428] Setting key 384d253d563c3d59726fd781beb9241f
I20260806 03:47:41.750378 12279 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260806 03:47:41.752125 26293 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 03:47:41.751678 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49278 (local address 127.11.253.254:44363)
0806 03:47:41.751716 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0806 03:47:41.751718 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:47:41.751867 (+ 149us) server_negotiation.cc:407] Connection header received
0806 03:47:41.751892 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:47:41.751893 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:47:41.751897 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:47:41.751943 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:47:41.752066 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49278: BlockingRecv error: recv got EOF from 127.0.0.1:49278 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
I20260806 03:47:41.752213 12279 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskToVWRe/build/release/bin/kudu with pid 26322
I20260806 03:47:41.770911 12279 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskToVWRe/build/release/bin/kudu with pid 26500
I20260806 03:47:41.787858 12279 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskToVWRe/build/release/bin/kudu with pid 26676
I20260806 03:47:41.795846 12279 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskToVWRe/build/release/bin/kudu with pid 26200
2026-08-06T03:47:41Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (597 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785988004637995-12279-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 03:47:41 dist-test-slave-6ntk krb5kdc[26863](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 03:47:41 dist-test-slave-6ntk krb5kdc[26863](info): set up 2 sockets
Aug 06 03:47:41 dist-test-slave-6ntk krb5kdc[26863](info): commencing operation
krb5kdc: starting...
W20260806 03:47:43.038206 12279 mini_kdc.cc:121] Time spent starting KDC: real 1.213s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785988004637995-12279-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785988004637995-12279-0/krb5kdc/test-user.keytab.
Aug 06 03:47:43 dist-test-slave-6ntk krb5kdc[26863](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988063, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T03:47:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T03:47:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.253.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.253.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.253.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.254.keytab.
Entry for principal kudu/127.11.253.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.254.keytab.
I20260806 03:47:43.166677 12279 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskToVWRe/build/release/bin/kudu
/tmp/dist-test-taskToVWRe/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785988004637995-12279-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785988004637995-12279-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785988004637995-12279-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785988004637995-12279-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.253.254:44091
--webserver_interface=127.11.253.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.254.keytab
--principal=kudu/127.11.253.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.253.212:41001
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.253.254:44091
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785988004637995-12279-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785988004637995-12279-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785988004637995-12279-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:47:43.250988 26883 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:47:43.251240 26883 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:47:43.251277 26883 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:47:43.252434 26883 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 03:47:43.252501 26883 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:47:43.252525 26883 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:47:43.252547 26883 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 03:47:43.252578 26883 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 03:47:43.254149 26883 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.253.212:41001
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785988004637995-12279-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785988004637995-12279-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.253.254:44091
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.254.keytab
--principal=kudu/127.11.253.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.253.254:44091
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785988004637995-12279-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.253.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26883
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785988004637995-12279-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:47:43.254438 26883 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:47:43.254673 26883 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:47:43.259730 26894 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:47:43.259732 26890 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:47:43.259716 26891 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:47:43.260030 26883 server_base.cc:1034] running on GCE node
I20260806 03:47:43.260272 26883 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:47:43.260557 26883 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:47:43.261730 26883 hybrid_clock.cc:648] HybridClock initialized: now 1785988063261707 us; error 36 us; skew 500 ppm
Aug 06 03:47:43 dist-test-slave-6ntk krb5kdc[26863](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988063, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.253.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:47:43.264420 26883 init.cc:377] Logged in from keytab as kudu/127.11.253.254@KRBTEST.COM (short username kudu)
I20260806 03:47:43.264811 26883 webserver.cc:460] Webserver started at http://127.11.253.254:45039/ using document root <none> and password file <none>
I20260806 03:47:43.265014 26883 fs_manager.cc:359] Metadata directory not provided
I20260806 03:47:43.265055 26883 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:47:43.265146 26883 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:47:43.266273 26883 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785988004637995-12279-0/minicluster-data/master-0/data/instance:
uuid: "57eb1d8313964593a93034ad0747c1ed"
format_stamp: "Formatted at 2026-08-06 03:47:43 on dist-test-slave-6ntk"
server_key: "c1b486aa6030faf1e1b68f379815e99a"
server_key_iv: "44d62c06e7fa37fb6e7dd1c3e01b2d07"
server_key_version: "encryptionkey@0"
I20260806 03:47:43.266593 26883 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785988004637995-12279-0/minicluster-data/master-0/wal/instance:
uuid: "57eb1d8313964593a93034ad0747c1ed"
format_stamp: "Formatted at 2026-08-06 03:47:43 on dist-test-slave-6ntk"
server_key: "c1b486aa6030faf1e1b68f379815e99a"
server_key_iv: "44d62c06e7fa37fb6e7dd1c3e01b2d07"
server_key_version: "encryptionkey@0"
I20260806 03:47:43.267879 26883 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 03:47:43.268589 26909 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:47:43.268774 26883 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 03:47:43.268887 26883 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785988004637995-12279-0/minicluster-data/master-0/data,/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785988004637995-12279-0/minicluster-data/master-0/wal
uuid: "57eb1d8313964593a93034ad0747c1ed"
format_stamp: "Formatted at 2026-08-06 03:47:43 on dist-test-slave-6ntk"
server_key: "c1b486aa6030faf1e1b68f379815e99a"
server_key_iv: "44d62c06e7fa37fb6e7dd1c3e01b2d07"
server_key_version: "encryptionkey@0"
I20260806 03:47:43.268965 26883 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785988004637995-12279-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785988004637995-12279-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785988004637995-12279-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:47:43.278779 26883 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:47:43.281148 26883 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:47:43.281327 26883 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:47:43.285445 26883 rpc_server.cc:307] RPC server started. Bound to: 127.11.253.254:44091
I20260806 03:47:43.285517 26988 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.253.254:44091 every 8 connection(s)
I20260806 03:47:43.285957 26883 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785988004637995-12279-0/minicluster-data/master-0/data/info.pb
I20260806 03:47:43.287014 26989 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:47:43.289330 26989 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 57eb1d8313964593a93034ad0747c1ed: Bootstrap starting.
I20260806 03:47:43.290020 26989 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 57eb1d8313964593a93034ad0747c1ed: Neither blocks nor log segments found. Creating new log.
I20260806 03:47:43.290320 26989 log.cc:824] T 00000000000000000000000000000000 P 57eb1d8313964593a93034ad0747c1ed: Log is configured to *not* fsync() on all Append() calls
I20260806 03:47:43.291046 26989 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 57eb1d8313964593a93034ad0747c1ed: No bootstrap required, opened a new log
I20260806 03:47:43.292028 12279 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskToVWRe/build/release/bin/kudu as pid 26883
I20260806 03:47:43.292141 12279 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785988004637995-12279-0/minicluster-data/master-0/wal/instance
I20260806 03:47:43.292316 12279 external_mini_cluster.cc:1428] Setting key eb9eac804a1ad0dbcb9ca51db23fc3b0
I20260806 03:47:43.293033 26989 raft_consensus.cc:348] T 00000000000000000000000000000000 P 57eb1d8313964593a93034ad0747c1ed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57eb1d8313964593a93034ad0747c1ed" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 44091 } }
I20260806 03:47:43.293223 26989 raft_consensus.cc:374] T 00000000000000000000000000000000 P 57eb1d8313964593a93034ad0747c1ed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:47:43.293264 26989 raft_consensus.cc:729] T 00000000000000000000000000000000 P 57eb1d8313964593a93034ad0747c1ed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57eb1d8313964593a93034ad0747c1ed, State: Initialized, Role: FOLLOWER
I20260806 03:47:43.293399 26989 consensus_queue.cc:260] T 00000000000000000000000000000000 P 57eb1d8313964593a93034ad0747c1ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57eb1d8313964593a93034ad0747c1ed" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 44091 } }
I20260806 03:47:43.293498 26989 raft_consensus.cc:388] T 00000000000000000000000000000000 P 57eb1d8313964593a93034ad0747c1ed [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 03:47:43.293541 26989 raft_consensus.cc:482] T 00000000000000000000000000000000 P 57eb1d8313964593a93034ad0747c1ed [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 03:47:43.293604 26989 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 57eb1d8313964593a93034ad0747c1ed [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:47:43.294361 26989 raft_consensus.cc:504] T 00000000000000000000000000000000 P 57eb1d8313964593a93034ad0747c1ed [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57eb1d8313964593a93034ad0747c1ed" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 44091 } }
I20260806 03:47:43.294472 26989 leader_election.cc:302] T 00000000000000000000000000000000 P 57eb1d8313964593a93034ad0747c1ed [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 57eb1d8313964593a93034ad0747c1ed; no voters:
I20260806 03:47:43.294633 26989 leader_election.cc:288] T 00000000000000000000000000000000 P 57eb1d8313964593a93034ad0747c1ed [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 03:47:43.294725 26995 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 57eb1d8313964593a93034ad0747c1ed [term 1 FOLLOWER]: Leader election won for term 1
I20260806 03:47:43.294879 26995 raft_consensus.cc:686] T 00000000000000000000000000000000 P 57eb1d8313964593a93034ad0747c1ed [term 1 LEADER]: Becoming Leader. State: Replica: 57eb1d8313964593a93034ad0747c1ed, State: Running, Role: LEADER
I20260806 03:47:43.294906 26989 sys_catalog.cc:564] T 00000000000000000000000000000000 P 57eb1d8313964593a93034ad0747c1ed [sys.catalog]: configured and running, proceeding with master startup.
I20260806 03:47:43.294968 26995 consensus_queue.cc:237] T 00000000000000000000000000000000 P 57eb1d8313964593a93034ad0747c1ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57eb1d8313964593a93034ad0747c1ed" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 44091 } }
I20260806 03:47:43.295312 26995 sys_catalog.cc:455] T 00000000000000000000000000000000 P 57eb1d8313964593a93034ad0747c1ed [sys.catalog]: SysCatalogTable state changed. Reason: New leader 57eb1d8313964593a93034ad0747c1ed. Latest consensus state: current_term: 1 leader_uuid: "57eb1d8313964593a93034ad0747c1ed" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57eb1d8313964593a93034ad0747c1ed" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 44091 } } }
I20260806 03:47:43.295397 26995 sys_catalog.cc:458] T 00000000000000000000000000000000 P 57eb1d8313964593a93034ad0747c1ed [sys.catalog]: This master's current role is: LEADER
I20260806 03:47:43.295357 26996 sys_catalog.cc:455] T 00000000000000000000000000000000 P 57eb1d8313964593a93034ad0747c1ed [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "57eb1d8313964593a93034ad0747c1ed" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57eb1d8313964593a93034ad0747c1ed" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 44091 } } }
I20260806 03:47:43.295560 26996 sys_catalog.cc:458] T 00000000000000000000000000000000 P 57eb1d8313964593a93034ad0747c1ed [sys.catalog]: This master's current role is: LEADER
W20260806 03:47:43.296653 27018 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 57eb1d8313964593a93034ad0747c1ed: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 03:47:43.296837 27018 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 03:47:43.296900 27007 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 03:47:43.297081 27007 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 03:47:43.298738 27007 catalog_manager.cc:1347] Generated new cluster ID: 2d6315f4b48e4282849652a68689654f
I20260806 03:47:43.298792 27007 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 03:47:43 dist-test-slave-6ntk krb5kdc[26863](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988063, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.253.254@KRBTEST.COM
I20260806 03:47:43.304139 26994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:43.292983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57490 (local address 127.11.253.254:44091)
0806 03:47:43.293191 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0806 03:47:43.293201 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 03:47:43.293238 (+ 37us) server_negotiation.cc:407] Connection header received
0806 03:47:43.293357 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:47:43.293365 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:47:43.293460 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:47:43.293609 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:47:43.294346 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:43.295775 (+ 1429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:43.296863 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:43.297407 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:43.297490 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:43.300235 (+ 2745us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:47:43.300240 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:47:43.300240 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:47:43.300253 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:47:43.302095 (+ 1842us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:47:43.302493 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:47:43.302496 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:47:43.302497 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:47:43.302602 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:47:43.302822 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:47:43.302823 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:47:43.302823 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:47:43.303067 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:47:43.303789 (+ 722us) server_negotiation.cc:1035] Waiting for connection context
0806 03:47:43.303853 (+ 64us) server_negotiation.cc:299] Negotiation successful
0806 03:47:43.303970 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149,"thread_start_us":95,"threads_started":1}
I20260806 03:47:43.313170 27007 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 03:47:43.313716 27007 catalog_manager.cc:1504] Loading token signing keys...
I20260806 03:47:43.315984 27007 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 57eb1d8313964593a93034ad0747c1ed: Generated new TSK 0
I20260806 03:47:43.316177 27007 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.253.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.253.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.253.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.193.keytab.
Entry for principal kudu/127.11.253.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.193.keytab.
I20260806 03:47:43.345429 12279 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskToVWRe/build/release/bin/kudu
/tmp/dist-test-taskToVWRe/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785988004637995-12279-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785988004637995-12279-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785988004637995-12279-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785988004637995-12279-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.253.193:0
--local_ip_for_outbound_sockets=127.11.253.193
--webserver_interface=127.11.253.193
--webserver_port=0
--tserver_master_addrs=127.11.253.254:44091
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.193.keytab
--principal=kudu/127.11.253.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.253.212:41001
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785988004637995-12279-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785988004637995-12279-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785988004637995-12279-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:47:43.430796 27025 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:47:43.431003 27025 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:47:43.431025 27025 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:47:43.432216 27025 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:47:43.432257 27025 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:47:43.432284 27025 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.253.193
I20260806 03:47:43.433681 27025 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.253.212:41001
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785988004637995-12279-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785988004637995-12279-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.193.keytab
--principal=kudu/127.11.253.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.253.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785988004637995-12279-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.253.193
--webserver_port=0
--tserver_master_addrs=127.11.253.254:44091
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27025
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.253.193
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785988004637995-12279-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:47:43.433919 27025 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:47:43.434140 27025 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:47:43.439149 27038 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:47:43.439275 27033 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:47:43.439158 27031 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:47:43.439416 27025 server_base.cc:1034] running on GCE node
I20260806 03:47:43.439618 27025 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:47:43.439864 27025 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:47:43.441028 27025 hybrid_clock.cc:648] HybridClock initialized: now 1785988063441024 us; error 38 us; skew 500 ppm
Aug 06 03:47:43 dist-test-slave-6ntk krb5kdc[26863](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988063, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.253.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:47:43.448103 27025 init.cc:377] Logged in from keytab as kudu/127.11.253.193@KRBTEST.COM (short username kudu)
I20260806 03:47:43.448571 27025 webserver.cc:460] Webserver started at http://127.11.253.193:34255/ using document root <none> and password file <none>
I20260806 03:47:43.448832 27025 fs_manager.cc:359] Metadata directory not provided
I20260806 03:47:43.448904 27025 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:47:43.449055 27025 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:47:43.450177 27025 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785988004637995-12279-0/minicluster-data/ts-0/data/instance:
uuid: "a0d650ff230b48daa19b4c199edc26f4"
format_stamp: "Formatted at 2026-08-06 03:47:43 on dist-test-slave-6ntk"
server_key: "57cf7225eb9b63d1a634eaa3aa05a788"
server_key_iv: "2ef907478037d30810dfad644d8ffa0a"
server_key_version: "encryptionkey@0"
I20260806 03:47:43.450531 27025 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785988004637995-12279-0/minicluster-data/ts-0/wal/instance:
uuid: "a0d650ff230b48daa19b4c199edc26f4"
format_stamp: "Formatted at 2026-08-06 03:47:43 on dist-test-slave-6ntk"
server_key: "57cf7225eb9b63d1a634eaa3aa05a788"
server_key_iv: "2ef907478037d30810dfad644d8ffa0a"
server_key_version: "encryptionkey@0"
I20260806 03:47:43.451925 27025 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 03:47:43.452673 27048 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:47:43.452862 27025 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 03:47:43.452970 27025 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785988004637995-12279-0/minicluster-data/ts-0/data,/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785988004637995-12279-0/minicluster-data/ts-0/wal
uuid: "a0d650ff230b48daa19b4c199edc26f4"
format_stamp: "Formatted at 2026-08-06 03:47:43 on dist-test-slave-6ntk"
server_key: "57cf7225eb9b63d1a634eaa3aa05a788"
server_key_iv: "2ef907478037d30810dfad644d8ffa0a"
server_key_version: "encryptionkey@0"
I20260806 03:47:43.453042 27025 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785988004637995-12279-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785988004637995-12279-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785988004637995-12279-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:47:43.491816 27025 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:47:43.494073 27025 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:47:43.494266 27025 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:47:43.494544 27025 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 03:47:43.494952 27025 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 03:47:43.495018 27025 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 03:47:43.495064 27025 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 03:47:43.495090 27025 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 03:47:43.501165 27025 rpc_server.cc:307] RPC server started. Bound to: 127.11.253.193:36455
I20260806 03:47:43.501243 27212 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.253.193:36455 every 8 connection(s)
I20260806 03:47:43.501622 27025 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785988004637995-12279-0/minicluster-data/ts-0/data/info.pb
Aug 06 03:47:43 dist-test-slave-6ntk krb5kdc[26863](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988063, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.253.193@KRBTEST.COM for kudu/127.11.253.254@KRBTEST.COM
I20260806 03:47:43.511763 12279 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskToVWRe/build/release/bin/kudu as pid 27025
I20260806 03:47:43.511902 12279 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785988004637995-12279-0/minicluster-data/ts-0/wal/instance
I20260806 03:47:43.512152 12279 external_mini_cluster.cc:1428] Setting key 7de5580fc1b149fb8c1ec089802f8da2
I20260806 03:47:43.512992 26994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:43.502681 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.253.193:58671 (local address 127.11.253.254:44091)
0806 03:47:43.502785 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0806 03:47:43.502788 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 03:47:43.503017 (+ 229us) server_negotiation.cc:407] Connection header received
0806 03:47:43.503699 (+ 682us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:47:43.503701 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:47:43.503718 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:47:43.503791 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:47:43.504916 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:43.505667 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:43.506839 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:43.507071 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:43.507157 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:43.509529 (+ 2372us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:47:43.509531 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:47:43.509532 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:47:43.509545 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:47:43.511087 (+ 1542us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:47:43.511590 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:47:43.511591 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:47:43.511592 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:47:43.511661 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:47:43.512015 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:47:43.512016 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:47:43.512016 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:47:43.512195 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:47:43.512349 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0806 03:47:43.512763 (+ 414us) server_negotiation.cc:299] Negotiation successful
0806 03:47:43.512898 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260806 03:47:43.513186 27217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:43.502619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:44091 (local address 127.11.253.193:58671)
0806 03:47:43.502849 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0806 03:47:43.502907 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0806 03:47:43.503549 (+ 642us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:47:43.503872 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:47:43.503876 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:47:43.504552 (+ 676us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:47:43.504785 (+ 233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:43.504790 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:43.505823 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:43.505824 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:43.506736 (+ 912us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:43.506739 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:43.507166 (+ 427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:43.507167 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:43.507311 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:43.507811 (+ 500us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:47:43.507826 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:47:43.509401 (+ 1575us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:47:43.511198 (+ 1797us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:47:43.511207 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:47:43.511210 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:47:43.511473 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:47:43.511740 (+ 267us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:47:43.511741 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:47:43.511741 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:47:43.511908 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:47:43.512385 (+ 477us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:47:43.512394 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:47:43.512577 (+ 183us) client_negotiation.cc:769] Sending connection context
0806 03:47:43.512698 (+ 121us) client_negotiation.cc:240] Negotiation successful
0806 03:47:43.512909 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":102,"threads_started":1}
I20260806 03:47:43.513974 27215 heartbeater.cc:347] Connected to a master server at 127.11.253.254:44091
I20260806 03:47:43.514096 27215 heartbeater.cc:464] Registering TS with master...
I20260806 03:47:43.514318 27215 heartbeater.cc:511] Master 127.11.253.254:44091 requested a full tablet report, sending...
I20260806 03:47:43.514952 26931 ts_manager.cc:194] Registered new tserver with Master: a0d650ff230b48daa19b4c199edc26f4 (127.11.253.193:36455)
I20260806 03:47:43.515925 26931 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.253.193@KRBTEST.COM'} at 127.11.253.193:58671
WARNING: no policy specified for kudu/127.11.253.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.253.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.253.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.194.keytab.
Entry for principal kudu/127.11.253.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.194.keytab.
I20260806 03:47:43.542197 12279 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskToVWRe/build/release/bin/kudu
/tmp/dist-test-taskToVWRe/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785988004637995-12279-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785988004637995-12279-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785988004637995-12279-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785988004637995-12279-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.253.194:0
--local_ip_for_outbound_sockets=127.11.253.194
--webserver_interface=127.11.253.194
--webserver_port=0
--tserver_master_addrs=127.11.253.254:44091
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.194.keytab
--principal=kudu/127.11.253.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.253.212:41001
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785988004637995-12279-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785988004637995-12279-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785988004637995-12279-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:47:43.627032 27222 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:47:43.627259 27222 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:47:43.627283 27222 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:47:43.628530 27222 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:47:43.628575 27222 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:47:43.628602 27222 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.253.194
I20260806 03:47:43.630125 27222 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.253.212:41001
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785988004637995-12279-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785988004637995-12279-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.194.keytab
--principal=kudu/127.11.253.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.253.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785988004637995-12279-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.253.194
--webserver_port=0
--tserver_master_addrs=127.11.253.254:44091
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27222
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.253.194
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785988004637995-12279-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:47:43.630329 27222 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:47:43.630534 27222 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:47:43.636004 27229 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:47:43.636035 27227 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:47:43.636013 27232 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:47:43.636188 27222 server_base.cc:1034] running on GCE node
I20260806 03:47:43.636523 27222 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:47:43.636826 27222 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:47:43.637939 27222 hybrid_clock.cc:648] HybridClock initialized: now 1785988063637917 us; error 40 us; skew 500 ppm
Aug 06 03:47:43 dist-test-slave-6ntk krb5kdc[26863](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988063, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.253.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:47:43.640565 27222 init.cc:377] Logged in from keytab as kudu/127.11.253.194@KRBTEST.COM (short username kudu)
I20260806 03:47:43.641070 27222 webserver.cc:460] Webserver started at http://127.11.253.194:39545/ using document root <none> and password file <none>
I20260806 03:47:43.641314 27222 fs_manager.cc:359] Metadata directory not provided
I20260806 03:47:43.641382 27222 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:47:43.641525 27222 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:47:43.642673 27222 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785988004637995-12279-0/minicluster-data/ts-1/data/instance:
uuid: "84d954cf7bce4867ad5f0a9f63b16b65"
format_stamp: "Formatted at 2026-08-06 03:47:43 on dist-test-slave-6ntk"
server_key: "ce76ec391d0644b2a03039e9e69839fc"
server_key_iv: "c55d39c0436f552d090007b376aa16ba"
server_key_version: "encryptionkey@0"
I20260806 03:47:43.643049 27222 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785988004637995-12279-0/minicluster-data/ts-1/wal/instance:
uuid: "84d954cf7bce4867ad5f0a9f63b16b65"
format_stamp: "Formatted at 2026-08-06 03:47:43 on dist-test-slave-6ntk"
server_key: "ce76ec391d0644b2a03039e9e69839fc"
server_key_iv: "c55d39c0436f552d090007b376aa16ba"
server_key_version: "encryptionkey@0"
I20260806 03:47:43.644479 27222 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260806 03:47:43.645331 27242 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:47:43.645560 27222 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 03:47:43.645665 27222 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785988004637995-12279-0/minicluster-data/ts-1/data,/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785988004637995-12279-0/minicluster-data/ts-1/wal
uuid: "84d954cf7bce4867ad5f0a9f63b16b65"
format_stamp: "Formatted at 2026-08-06 03:47:43 on dist-test-slave-6ntk"
server_key: "ce76ec391d0644b2a03039e9e69839fc"
server_key_iv: "c55d39c0436f552d090007b376aa16ba"
server_key_version: "encryptionkey@0"
I20260806 03:47:43.645752 27222 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785988004637995-12279-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785988004637995-12279-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785988004637995-12279-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:47:43.671607 27222 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:47:43.674140 27222 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:47:43.674322 27222 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:47:43.674597 27222 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 03:47:43.674948 27222 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 03:47:43.675010 27222 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 03:47:43.675045 27222 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 03:47:43.675077 27222 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 03:47:43.681064 27222 rpc_server.cc:307] RPC server started. Bound to: 127.11.253.194:37893
I20260806 03:47:43.681128 27404 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.253.194:37893 every 8 connection(s)
I20260806 03:47:43.681499 27222 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785988004637995-12279-0/minicluster-data/ts-1/data/info.pb
Aug 06 03:47:43 dist-test-slave-6ntk krb5kdc[26863](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988063, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.253.194@KRBTEST.COM for kudu/127.11.253.254@KRBTEST.COM
I20260806 03:47:43.688560 12279 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskToVWRe/build/release/bin/kudu as pid 27222
I20260806 03:47:43.688637 12279 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785988004637995-12279-0/minicluster-data/ts-1/wal/instance
I20260806 03:47:43.688836 12279 external_mini_cluster.cc:1428] Setting key e45cc613372c6e988a1a13c3ccb213d6
I20260806 03:47:43.691278 26994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:43.682379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.253.194:40607 (local address 127.11.253.254:44091)
0806 03:47:43.682434 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0806 03:47:43.682436 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:47:43.682631 (+ 195us) server_negotiation.cc:407] Connection header received
0806 03:47:43.683111 (+ 480us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:47:43.683112 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:47:43.683122 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:47:43.683185 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:47:43.684122 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:43.684611 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:43.685583 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:43.685785 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:43.685843 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:43.688037 (+ 2194us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:47:43.688038 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:47:43.688038 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:47:43.688045 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:47:43.689683 (+ 1638us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:47:43.690173 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:47:43.690174 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:47:43.690174 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:47:43.690241 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:47:43.690520 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:47:43.690521 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:47:43.690522 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:47:43.690646 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:47:43.690719 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0806 03:47:43.691061 (+ 342us) server_negotiation.cc:299] Negotiation successful
0806 03:47:43.691154 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260806 03:47:43.691900 27409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:43.682329 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:44091 (local address 127.11.253.194:40607)
0806 03:47:43.682493 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0806 03:47:43.682538 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 03:47:43.683025 (+ 487us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:47:43.683238 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:47:43.683242 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:47:43.683832 (+ 590us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:47:43.684034 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:43.684039 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:43.684698 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:43.684699 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:43.685492 (+ 793us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:43.685494 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:43.685864 (+ 370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:43.685865 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:43.686043 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:43.686562 (+ 519us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:47:43.686571 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:47:43.687951 (+ 1380us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:47:43.689781 (+ 1830us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:47:43.689790 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:47:43.689792 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:47:43.690090 (+ 298us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:47:43.690313 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:47:43.690314 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:47:43.690314 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:47:43.690446 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:47:43.690723 (+ 277us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:47:43.690731 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:47:43.690898 (+ 167us) client_negotiation.cc:769] Sending connection context
0806 03:47:43.691435 (+ 537us) client_negotiation.cc:240] Negotiation successful
0806 03:47:43.691634 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":87,"threads_started":1}
I20260806 03:47:43.692390 27405 heartbeater.cc:347] Connected to a master server at 127.11.253.254:44091
I20260806 03:47:43.692539 27405 heartbeater.cc:464] Registering TS with master...
I20260806 03:47:43.692754 27405 heartbeater.cc:511] Master 127.11.253.254:44091 requested a full tablet report, sending...
I20260806 03:47:43.693224 26931 ts_manager.cc:194] Registered new tserver with Master: 84d954cf7bce4867ad5f0a9f63b16b65 (127.11.253.194:37893)
I20260806 03:47:43.693751 26931 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.253.194@KRBTEST.COM'} at 127.11.253.194:40607
WARNING: no policy specified for kudu/127.11.253.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.253.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.253.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.195.keytab.
Entry for principal kudu/127.11.253.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.195.keytab.
I20260806 03:47:43.717840 12279 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskToVWRe/build/release/bin/kudu
/tmp/dist-test-taskToVWRe/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785988004637995-12279-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785988004637995-12279-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785988004637995-12279-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785988004637995-12279-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.253.195:0
--local_ip_for_outbound_sockets=127.11.253.195
--webserver_interface=127.11.253.195
--webserver_port=0
--tserver_master_addrs=127.11.253.254:44091
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.195.keytab
--principal=kudu/127.11.253.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.253.212:41001
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785988004637995-12279-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785988004637995-12279-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785988004637995-12279-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:47:43.803113 27412 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:47:43.803336 27412 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:47:43.803361 27412 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:47:43.804528 27412 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:47:43.804570 27412 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:47:43.804598 27412 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.253.195
I20260806 03:47:43.806170 27412 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.253.212:41001
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785988004637995-12279-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785988004637995-12279-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.195.keytab
--principal=kudu/127.11.253.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.253.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785988004637995-12279-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.253.195
--webserver_port=0
--tserver_master_addrs=127.11.253.254:44091
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27412
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.253.195
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785988004637995-12279-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:47:43.806433 27412 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:47:43.806685 27412 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:47:43.811904 27424 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:47:43.811834 27426 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:47:43.811946 27422 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:47:43.811995 27412 server_base.cc:1034] running on GCE node
I20260806 03:47:43.812225 27412 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:47:43.812510 27412 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:47:43.813668 27412 hybrid_clock.cc:648] HybridClock initialized: now 1785988063813666 us; error 40 us; skew 500 ppm
Aug 06 03:47:43 dist-test-slave-6ntk krb5kdc[26863](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988063, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.253.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:47:43.816216 27412 init.cc:377] Logged in from keytab as kudu/127.11.253.195@KRBTEST.COM (short username kudu)
I20260806 03:47:43.816648 27412 webserver.cc:460] Webserver started at http://127.11.253.195:39741/ using document root <none> and password file <none>
I20260806 03:47:43.816890 27412 fs_manager.cc:359] Metadata directory not provided
I20260806 03:47:43.816959 27412 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:47:43.817119 27412 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:47:43.818274 27412 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785988004637995-12279-0/minicluster-data/ts-2/data/instance:
uuid: "8178896c2ac3402f818b46b702589893"
format_stamp: "Formatted at 2026-08-06 03:47:43 on dist-test-slave-6ntk"
server_key: "6f9809e0034e149bfad7c028d6c7ae05"
server_key_iv: "8c14afea60472fcb2d53f69f6e57d9c7"
server_key_version: "encryptionkey@0"
I20260806 03:47:43.818652 27412 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785988004637995-12279-0/minicluster-data/ts-2/wal/instance:
uuid: "8178896c2ac3402f818b46b702589893"
format_stamp: "Formatted at 2026-08-06 03:47:43 on dist-test-slave-6ntk"
server_key: "6f9809e0034e149bfad7c028d6c7ae05"
server_key_iv: "8c14afea60472fcb2d53f69f6e57d9c7"
server_key_version: "encryptionkey@0"
I20260806 03:47:43.820010 27412 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 03:47:43.820796 27438 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:47:43.820972 27412 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 03:47:43.821084 27412 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785988004637995-12279-0/minicluster-data/ts-2/data,/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785988004637995-12279-0/minicluster-data/ts-2/wal
uuid: "8178896c2ac3402f818b46b702589893"
format_stamp: "Formatted at 2026-08-06 03:47:43 on dist-test-slave-6ntk"
server_key: "6f9809e0034e149bfad7c028d6c7ae05"
server_key_iv: "8c14afea60472fcb2d53f69f6e57d9c7"
server_key_version: "encryptionkey@0"
I20260806 03:47:43.821161 27412 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785988004637995-12279-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785988004637995-12279-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785988004637995-12279-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:47:43.838865 27412 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:47:43.841482 27412 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:47:43.841682 27412 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:47:43.842025 27412 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 03:47:43.842437 27412 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 03:47:43.842479 27412 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 03:47:43.842549 27412 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 03:47:43.842586 27412 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 03:47:43.849812 27412 rpc_server.cc:307] RPC server started. Bound to: 127.11.253.195:45675
I20260806 03:47:43.849898 27593 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.253.195:45675 every 8 connection(s)
I20260806 03:47:43.850318 27412 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785988004637995-12279-0/minicluster-data/ts-2/data/info.pb
I20260806 03:47:43.853506 12279 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskToVWRe/build/release/bin/kudu as pid 27412
I20260806 03:47:43.853586 12279 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785988004637995-12279-0/minicluster-data/ts-2/wal/instance
I20260806 03:47:43.853780 12279 external_mini_cluster.cc:1428] Setting key 45b223ca29643eb1d0fdea02fced842f
Aug 06 03:47:43 dist-test-slave-6ntk krb5kdc[26863](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988063, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.253.195@KRBTEST.COM for kudu/127.11.253.254@KRBTEST.COM
I20260806 03:47:43.860327 26994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:43.851202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.253.195:52771 (local address 127.11.253.254:44091)
0806 03:47:43.851302 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0806 03:47:43.851304 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:47:43.851542 (+ 238us) server_negotiation.cc:407] Connection header received
0806 03:47:43.852066 (+ 524us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:47:43.852067 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:47:43.852078 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:47:43.852138 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:47:43.853079 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:43.853616 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:43.855014 (+ 1398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:43.855278 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:43.855368 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:43.857579 (+ 2211us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:47:43.857581 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:47:43.857581 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:47:43.857589 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:47:43.858874 (+ 1285us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:47:43.859299 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:47:43.859301 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:47:43.859301 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:47:43.859374 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:47:43.859635 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:47:43.859636 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:47:43.859636 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:47:43.859759 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:47:43.859835 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0806 03:47:43.860145 (+ 310us) server_negotiation.cc:299] Negotiation successful
0806 03:47:43.860232 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260806 03:47:43.860507 27596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:43.851169 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:44091 (local address 127.11.253.195:52771)
0806 03:47:43.851408 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0806 03:47:43.851446 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0806 03:47:43.851942 (+ 496us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:47:43.852194 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:47:43.852197 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:47:43.852792 (+ 595us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:47:43.852995 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:43.853000 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:43.853709 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:43.853711 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:43.854918 (+ 1207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:43.854921 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:43.855401 (+ 480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:43.855402 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:43.855537 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:43.856033 (+ 496us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:47:43.856042 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:47:43.857476 (+ 1434us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:47:43.858972 (+ 1496us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:47:43.858979 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:47:43.858981 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:47:43.859218 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:47:43.859456 (+ 238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:47:43.859457 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:47:43.859457 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:47:43.859566 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:47:43.859839 (+ 273us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:47:43.859857 (+ 18us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:47:43.860004 (+ 147us) client_negotiation.cc:769] Sending connection context
0806 03:47:43.860109 (+ 105us) client_negotiation.cc:240] Negotiation successful
0806 03:47:43.860269 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":139,"threads_started":1}
I20260806 03:47:43.861014 27594 heartbeater.cc:347] Connected to a master server at 127.11.253.254:44091
I20260806 03:47:43.861102 27594 heartbeater.cc:464] Registering TS with master...
I20260806 03:47:43.861258 27594 heartbeater.cc:511] Master 127.11.253.254:44091 requested a full tablet report, sending...
I20260806 03:47:43.861636 26931 ts_manager.cc:194] Registered new tserver with Master: 8178896c2ac3402f818b46b702589893 (127.11.253.195:45675)
I20260806 03:47:43.862109 26931 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.253.195@KRBTEST.COM'} at 127.11.253.195:52771
I20260806 03:47:43.865510 12279 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 03:47:43.873507 26994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:43.867045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57496 (local address 127.11.253.254:44091)
0806 03:47:43.867102 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0806 03:47:43.867104 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:47:43.867113 (+ 9us) server_negotiation.cc:407] Connection header received
0806 03:47:43.867189 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:47:43.867190 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:47:43.867202 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:47:43.867270 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:47:43.867837 (+ 567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:43.868316 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:43.869063 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:43.869250 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:43.869325 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:43.870658 (+ 1333us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:47:43.870659 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:47:43.870660 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:47:43.870666 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:47:43.872172 (+ 1506us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:47:43.872648 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:47:43.872649 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:47:43.872649 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:47:43.872710 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:47:43.872918 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:47:43.872919 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:47:43.872920 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:47:43.873089 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:47:43.873211 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0806 03:47:43.873312 (+ 101us) server_negotiation.cc:299] Negotiation successful
0806 03:47:43.873398 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260806 03:47:43.875396 26931 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57496:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 03:47:43.877084 26931 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 03:47:43.883767 27615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:43.879564 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.194:37893 (local address 127.0.0.1:45792)
0806 03:47:43.880060 (+ 496us) negotiation.cc:107] Waiting for socket to connect
0806 03:47:43.880070 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 03:47:43.880108 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:47:43.880463 (+ 355us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:47:43.880464 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:47:43.880466 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 03:47:43.880512 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:43.880514 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:43.881444 (+ 930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:43.881445 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:43.882635 (+ 1190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:43.882638 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:43.883450 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:43.883451 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:43.883533 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:43.883548 (+ 15us) client_negotiation.cc:769] Sending connection context
0806 03:47:43.883626 (+ 78us) client_negotiation.cc:240] Negotiation successful
0806 03:47:43.883678 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":478,"thread_start_us":112,"threads_started":1}
I20260806 03:47:43.883800 27617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:43.879668 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45792 (local address 127.11.253.194:37893)
0806 03:47:43.880282 (+ 614us) server_negotiation.cc:206] Beginning negotiation
0806 03:47:43.880286 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 03:47:43.880294 (+ 8us) server_negotiation.cc:407] Connection header received
0806 03:47:43.880360 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:47:43.880363 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:47:43.880379 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:47:43.880452 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 03:47:43.880567 (+ 115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:43.881338 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:43.882712 (+ 1374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:43.883357 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:43.883436 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:43.883495 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0806 03:47:43.883657 (+ 162us) server_negotiation.cc:299] Negotiation successful
0806 03:47:43.883704 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":584,"thread_start_us":104,"threads_started":1}
I20260806 03:47:43.884111 27612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:43.879560 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.195:45675 (local address 127.0.0.1:35876)
0806 03:47:43.879834 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0806 03:47:43.879883 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0806 03:47:43.879937 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:47:43.880223 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:47:43.880226 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:47:43.880229 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 03:47:43.880316 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:43.880318 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:43.881956 (+ 1638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:43.881957 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:43.882938 (+ 981us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:43.882940 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:43.883896 (+ 956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:43.883897 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:43.883959 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:43.883963 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 03:47:43.883992 (+ 29us) client_negotiation.cc:240] Negotiation successful
0806 03:47:43.884034 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":248,"thread_start_us":111,"threads_started":1}
I20260806 03:47:43.884455 27618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:43.879841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35876 (local address 127.11.253.195:45675)
0806 03:47:43.880046 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0806 03:47:43.880051 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 03:47:43.880061 (+ 10us) server_negotiation.cc:407] Connection header received
0806 03:47:43.880103 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:47:43.880105 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:47:43.880117 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:47:43.880205 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 03:47:43.880672 (+ 467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:43.881808 (+ 1136us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:43.883043 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:43.883772 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:43.884230 (+ 458us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:43.884298 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0806 03:47:43.884327 (+ 29us) server_negotiation.cc:299] Negotiation successful
0806 03:47:43.884378 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":166,"thread_start_us":97,"threads_started":1}
I20260806 03:47:43.884755 27318 tablet_service.cc:1467] Processing CreateTablet for tablet b4e0cb30ba7a42a38c7537240ecf169f (DEFAULT_TABLE table=test-table [id=30fdfa43eddc46ce9e8d80a0197395e5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 03:47:43.885187 27318 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b4e0cb30ba7a42a38c7537240ecf169f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:47:43.885128 27499 tablet_service.cc:1467] Processing CreateTablet for tablet b4e0cb30ba7a42a38c7537240ecf169f (DEFAULT_TABLE table=test-table [id=30fdfa43eddc46ce9e8d80a0197395e5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 03:47:43.885329 27614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:43.879560 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.193:36455 (local address 127.0.0.1:52696)
0806 03:47:43.879875 (+ 315us) negotiation.cc:107] Waiting for socket to connect
0806 03:47:43.879887 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 03:47:43.879948 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:47:43.880635 (+ 687us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:47:43.880636 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:47:43.880638 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 03:47:43.880675 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:43.880676 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:43.883025 (+ 2349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:43.883025 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:43.884073 (+ 1048us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:43.884076 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:43.885068 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:43.885069 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:43.885137 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:43.885141 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 03:47:43.885204 (+ 63us) client_negotiation.cc:240] Negotiation successful
0806 03:47:43.885246 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":290,"mutex_wait_us":3,"thread_start_us":56,"threads_started":1}
I20260806 03:47:43.885383 27619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:43.880053 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52696 (local address 127.11.253.193:36455)
0806 03:47:43.880490 (+ 437us) server_negotiation.cc:206] Beginning negotiation
0806 03:47:43.880495 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 03:47:43.880505 (+ 10us) server_negotiation.cc:407] Connection header received
0806 03:47:43.880543 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:47:43.880546 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:47:43.880562 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:47:43.880642 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 03:47:43.880721 (+ 79us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:43.881812 (+ 1091us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:43.884160 (+ 2348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:43.884900 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:43.884979 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:43.885031 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0806 03:47:43.885249 (+ 218us) server_negotiation.cc:299] Negotiation successful
0806 03:47:43.885300 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":396,"thread_start_us":103,"threads_started":1}
I20260806 03:47:43.885481 27499 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b4e0cb30ba7a42a38c7537240ecf169f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:47:43.886597 27120 tablet_service.cc:1467] Processing CreateTablet for tablet b4e0cb30ba7a42a38c7537240ecf169f (DEFAULT_TABLE table=test-table [id=30fdfa43eddc46ce9e8d80a0197395e5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 03:47:43.886958 27120 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b4e0cb30ba7a42a38c7537240ecf169f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:47:43.888826 27620 tablet_bootstrap.cc:492] T b4e0cb30ba7a42a38c7537240ecf169f P 84d954cf7bce4867ad5f0a9f63b16b65: Bootstrap starting.
I20260806 03:47:43.889140 27621 tablet_bootstrap.cc:492] T b4e0cb30ba7a42a38c7537240ecf169f P 8178896c2ac3402f818b46b702589893: Bootstrap starting.
I20260806 03:47:43.889665 27620 tablet_bootstrap.cc:654] T b4e0cb30ba7a42a38c7537240ecf169f P 84d954cf7bce4867ad5f0a9f63b16b65: Neither blocks nor log segments found. Creating new log.
I20260806 03:47:43.889830 27621 tablet_bootstrap.cc:654] T b4e0cb30ba7a42a38c7537240ecf169f P 8178896c2ac3402f818b46b702589893: Neither blocks nor log segments found. Creating new log.
I20260806 03:47:43.890352 27620 log.cc:824] T b4e0cb30ba7a42a38c7537240ecf169f P 84d954cf7bce4867ad5f0a9f63b16b65: Log is configured to *not* fsync() on all Append() calls
I20260806 03:47:43.890452 27621 log.cc:824] T b4e0cb30ba7a42a38c7537240ecf169f P 8178896c2ac3402f818b46b702589893: Log is configured to *not* fsync() on all Append() calls
I20260806 03:47:43.890766 27622 tablet_bootstrap.cc:492] T b4e0cb30ba7a42a38c7537240ecf169f P a0d650ff230b48daa19b4c199edc26f4: Bootstrap starting.
I20260806 03:47:43.891252 27620 tablet_bootstrap.cc:492] T b4e0cb30ba7a42a38c7537240ecf169f P 84d954cf7bce4867ad5f0a9f63b16b65: No bootstrap required, opened a new log
I20260806 03:47:43.891260 27621 tablet_bootstrap.cc:492] T b4e0cb30ba7a42a38c7537240ecf169f P 8178896c2ac3402f818b46b702589893: No bootstrap required, opened a new log
I20260806 03:47:43.891345 27620 ts_tablet_manager.cc:1397] T b4e0cb30ba7a42a38c7537240ecf169f P 84d954cf7bce4867ad5f0a9f63b16b65: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 03:47:43.891392 27621 ts_tablet_manager.cc:1397] T b4e0cb30ba7a42a38c7537240ecf169f P 8178896c2ac3402f818b46b702589893: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260806 03:47:43.891527 27622 tablet_bootstrap.cc:654] T b4e0cb30ba7a42a38c7537240ecf169f P a0d650ff230b48daa19b4c199edc26f4: Neither blocks nor log segments found. Creating new log.
I20260806 03:47:43.892107 27622 log.cc:824] T b4e0cb30ba7a42a38c7537240ecf169f P a0d650ff230b48daa19b4c199edc26f4: Log is configured to *not* fsync() on all Append() calls
I20260806 03:47:43.892910 27622 tablet_bootstrap.cc:492] T b4e0cb30ba7a42a38c7537240ecf169f P a0d650ff230b48daa19b4c199edc26f4: No bootstrap required, opened a new log
I20260806 03:47:43.893005 27622 ts_tablet_manager.cc:1397] T b4e0cb30ba7a42a38c7537240ecf169f P a0d650ff230b48daa19b4c199edc26f4: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 03:47:43.893357 27620 raft_consensus.cc:348] T b4e0cb30ba7a42a38c7537240ecf169f P 84d954cf7bce4867ad5f0a9f63b16b65 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8178896c2ac3402f818b46b702589893" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 45675 } } peers { permanent_uuid: "84d954cf7bce4867ad5f0a9f63b16b65" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 37893 } } peers { permanent_uuid: "a0d650ff230b48daa19b4c199edc26f4" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 36455 } }
I20260806 03:47:43.893358 27621 raft_consensus.cc:348] T b4e0cb30ba7a42a38c7537240ecf169f P 8178896c2ac3402f818b46b702589893 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8178896c2ac3402f818b46b702589893" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 45675 } } peers { permanent_uuid: "84d954cf7bce4867ad5f0a9f63b16b65" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 37893 } } peers { permanent_uuid: "a0d650ff230b48daa19b4c199edc26f4" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 36455 } }
I20260806 03:47:43.893540 27621 raft_consensus.cc:374] T b4e0cb30ba7a42a38c7537240ecf169f P 8178896c2ac3402f818b46b702589893 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:47:43.893540 27620 raft_consensus.cc:374] T b4e0cb30ba7a42a38c7537240ecf169f P 84d954cf7bce4867ad5f0a9f63b16b65 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:47:43.893577 27621 raft_consensus.cc:729] T b4e0cb30ba7a42a38c7537240ecf169f P 8178896c2ac3402f818b46b702589893 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8178896c2ac3402f818b46b702589893, State: Initialized, Role: FOLLOWER
I20260806 03:47:43.893577 27620 raft_consensus.cc:729] T b4e0cb30ba7a42a38c7537240ecf169f P 84d954cf7bce4867ad5f0a9f63b16b65 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84d954cf7bce4867ad5f0a9f63b16b65, State: Initialized, Role: FOLLOWER
I20260806 03:47:43.893704 27620 consensus_queue.cc:260] T b4e0cb30ba7a42a38c7537240ecf169f P 84d954cf7bce4867ad5f0a9f63b16b65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8178896c2ac3402f818b46b702589893" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 45675 } } peers { permanent_uuid: "84d954cf7bce4867ad5f0a9f63b16b65" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 37893 } } peers { permanent_uuid: "a0d650ff230b48daa19b4c199edc26f4" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 36455 } }
I20260806 03:47:43.893704 27621 consensus_queue.cc:260] T b4e0cb30ba7a42a38c7537240ecf169f P 8178896c2ac3402f818b46b702589893 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8178896c2ac3402f818b46b702589893" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 45675 } } peers { permanent_uuid: "84d954cf7bce4867ad5f0a9f63b16b65" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 37893 } } peers { permanent_uuid: "a0d650ff230b48daa19b4c199edc26f4" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 36455 } }
I20260806 03:47:43.894045 27621 ts_tablet_manager.cc:1428] T b4e0cb30ba7a42a38c7537240ecf169f P 8178896c2ac3402f818b46b702589893: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260806 03:47:43.894045 27620 ts_tablet_manager.cc:1428] T b4e0cb30ba7a42a38c7537240ecf169f P 84d954cf7bce4867ad5f0a9f63b16b65: Time spent starting tablet: real 0.003s user 0.001s sys 0.002s
I20260806 03:47:43.894065 27405 heartbeater.cc:503] Master 127.11.253.254:44091 was elected leader, sending a full tablet report...
I20260806 03:47:43.894174 27594 heartbeater.cc:503] Master 127.11.253.254:44091 was elected leader, sending a full tablet report...
I20260806 03:47:43.895504 27622 raft_consensus.cc:348] T b4e0cb30ba7a42a38c7537240ecf169f P a0d650ff230b48daa19b4c199edc26f4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8178896c2ac3402f818b46b702589893" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 45675 } } peers { permanent_uuid: "84d954cf7bce4867ad5f0a9f63b16b65" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 37893 } } peers { permanent_uuid: "a0d650ff230b48daa19b4c199edc26f4" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 36455 } }
I20260806 03:47:43.895691 27622 raft_consensus.cc:374] T b4e0cb30ba7a42a38c7537240ecf169f P a0d650ff230b48daa19b4c199edc26f4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:47:43.895749 27622 raft_consensus.cc:729] T b4e0cb30ba7a42a38c7537240ecf169f P a0d650ff230b48daa19b4c199edc26f4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0d650ff230b48daa19b4c199edc26f4, State: Initialized, Role: FOLLOWER
I20260806 03:47:43.895951 27622 consensus_queue.cc:260] T b4e0cb30ba7a42a38c7537240ecf169f P a0d650ff230b48daa19b4c199edc26f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8178896c2ac3402f818b46b702589893" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 45675 } } peers { permanent_uuid: "84d954cf7bce4867ad5f0a9f63b16b65" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 37893 } } peers { permanent_uuid: "a0d650ff230b48daa19b4c199edc26f4" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 36455 } }
I20260806 03:47:43.896266 27622 ts_tablet_manager.cc:1428] T b4e0cb30ba7a42a38c7537240ecf169f P a0d650ff230b48daa19b4c199edc26f4: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 03:47:43.896314 27215 heartbeater.cc:503] Master 127.11.253.254:44091 was elected leader, sending a full tablet report...
W20260806 03:47:43.932174 27407 tablet.cc:2369] T b4e0cb30ba7a42a38c7537240ecf169f P 84d954cf7bce4867ad5f0a9f63b16b65: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 03:47:44.002471 27216 tablet.cc:2369] T b4e0cb30ba7a42a38c7537240ecf169f P a0d650ff230b48daa19b4c199edc26f4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 03:47:44.037727 27631 raft_consensus.cc:482] T b4e0cb30ba7a42a38c7537240ecf169f P 8178896c2ac3402f818b46b702589893 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 03:47:44.037854 27631 raft_consensus.cc:504] T b4e0cb30ba7a42a38c7537240ecf169f P 8178896c2ac3402f818b46b702589893 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8178896c2ac3402f818b46b702589893" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 45675 } } peers { permanent_uuid: "84d954cf7bce4867ad5f0a9f63b16b65" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 37893 } } peers { permanent_uuid: "a0d650ff230b48daa19b4c199edc26f4" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 36455 } }
I20260806 03:47:44.038316 27631 leader_election.cc:288] T b4e0cb30ba7a42a38c7537240ecf169f P 8178896c2ac3402f818b46b702589893 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84d954cf7bce4867ad5f0a9f63b16b65 (127.11.253.194:37893), a0d650ff230b48daa19b4c199edc26f4 (127.11.253.193:36455)
I20260806 03:47:44.042593 27619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:44.038577 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.253.195:34351 (local address 127.11.253.193:36455)
0806 03:47:44.038626 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0806 03:47:44.038628 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:47:44.038816 (+ 188us) server_negotiation.cc:407] Connection header received
0806 03:47:44.038874 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:47:44.038875 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:47:44.038886 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:47:44.038938 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 03:47:44.039108 (+ 170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:44.039794 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:44.041547 (+ 1753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:44.042146 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:44.042227 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:44.042236 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0806 03:47:44.042478 (+ 242us) server_negotiation.cc:299] Negotiation successful
0806 03:47:44.042518 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260806 03:47:44.042593 27617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:44.038628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.253.195:60209 (local address 127.11.253.194:37893)
0806 03:47:44.038691 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0806 03:47:44.038693 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:47:44.038703 (+ 10us) server_negotiation.cc:407] Connection header received
0806 03:47:44.038740 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:47:44.038741 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:47:44.038749 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:47:44.038830 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 03:47:44.039035 (+ 205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:44.039815 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:44.041547 (+ 1732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:44.042146 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:44.042227 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:44.042236 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0806 03:47:44.042480 (+ 244us) server_negotiation.cc:299] Negotiation successful
0806 03:47:44.042518 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260806 03:47:44.042661 27596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:44.038508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.194:37893 (local address 127.11.253.195:60209)
0806 03:47:44.038575 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0806 03:47:44.038587 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 03:47:44.038636 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:47:44.038837 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:47:44.038838 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:47:44.038840 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 03:47:44.038940 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:44.038943 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:44.039910 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:44.039912 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:44.041440 (+ 1528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:44.041443 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:44.042282 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:44.042284 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:44.042402 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:44.042406 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 03:47:44.042464 (+ 58us) client_negotiation.cc:240] Negotiation successful
0806 03:47:44.042508 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45,"mutex_wait_us":62}
I20260806 03:47:44.042894 27635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:44.038517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.193:36455 (local address 127.11.253.195:34351)
0806 03:47:44.038730 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0806 03:47:44.038741 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 03:47:44.038808 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:47:44.038966 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:47:44.038967 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:47:44.038969 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 03:47:44.039034 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:44.039035 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:44.039910 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:44.039912 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:44.041459 (+ 1547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:47:44.041461 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:47:44.042283 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:47:44.042284 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:47:44.042377 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:44.042382 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 03:47:44.042454 (+ 72us) client_negotiation.cc:240] Negotiation successful
0806 03:47:44.042508 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":192,"mutex_wait_us":117,"thread_start_us":129,"threads_started":1}
I20260806 03:47:44.043128 27340 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b4e0cb30ba7a42a38c7537240ecf169f" candidate_uuid: "8178896c2ac3402f818b46b702589893" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84d954cf7bce4867ad5f0a9f63b16b65" is_pre_election: true
I20260806 03:47:44.043322 27340 raft_consensus.cc:2454] T b4e0cb30ba7a42a38c7537240ecf169f P 84d954cf7bce4867ad5f0a9f63b16b65 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8178896c2ac3402f818b46b702589893 in term 0.
I20260806 03:47:44.043365 27147 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b4e0cb30ba7a42a38c7537240ecf169f" candidate_uuid: "8178896c2ac3402f818b46b702589893" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0d650ff230b48daa19b4c199edc26f4" is_pre_election: true
I20260806 03:47:44.043550 27147 raft_consensus.cc:2454] T b4e0cb30ba7a42a38c7537240ecf169f P a0d650ff230b48daa19b4c199edc26f4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8178896c2ac3402f818b46b702589893 in term 0.
I20260806 03:47:44.043620 27444 leader_election.cc:302] T b4e0cb30ba7a42a38c7537240ecf169f P 8178896c2ac3402f818b46b702589893 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8178896c2ac3402f818b46b702589893, 84d954cf7bce4867ad5f0a9f63b16b65; no voters:
I20260806 03:47:44.043828 27631 raft_consensus.cc:2781] T b4e0cb30ba7a42a38c7537240ecf169f P 8178896c2ac3402f818b46b702589893 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 03:47:44.043898 27631 raft_consensus.cc:482] T b4e0cb30ba7a42a38c7537240ecf169f P 8178896c2ac3402f818b46b702589893 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 03:47:44.043915 27631 raft_consensus.cc:3037] T b4e0cb30ba7a42a38c7537240ecf169f P 8178896c2ac3402f818b46b702589893 [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:47:44.044684 27631 raft_consensus.cc:504] T b4e0cb30ba7a42a38c7537240ecf169f P 8178896c2ac3402f818b46b702589893 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8178896c2ac3402f818b46b702589893" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 45675 } } peers { permanent_uuid: "84d954cf7bce4867ad5f0a9f63b16b65" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 37893 } } peers { permanent_uuid: "a0d650ff230b48daa19b4c199edc26f4" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 36455 } }
I20260806 03:47:44.044850 27631 leader_election.cc:288] T b4e0cb30ba7a42a38c7537240ecf169f P 8178896c2ac3402f818b46b702589893 [CANDIDATE]: Term 1 election: Requested vote from peers 84d954cf7bce4867ad5f0a9f63b16b65 (127.11.253.194:37893), a0d650ff230b48daa19b4c199edc26f4 (127.11.253.193:36455)
I20260806 03:47:44.045015 27340 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b4e0cb30ba7a42a38c7537240ecf169f" candidate_uuid: "8178896c2ac3402f818b46b702589893" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84d954cf7bce4867ad5f0a9f63b16b65"
I20260806 03:47:44.045060 27147 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b4e0cb30ba7a42a38c7537240ecf169f" candidate_uuid: "8178896c2ac3402f818b46b702589893" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0d650ff230b48daa19b4c199edc26f4"
I20260806 03:47:44.045095 27340 raft_consensus.cc:3037] T b4e0cb30ba7a42a38c7537240ecf169f P 84d954cf7bce4867ad5f0a9f63b16b65 [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:47:44.045172 27147 raft_consensus.cc:3037] T b4e0cb30ba7a42a38c7537240ecf169f P a0d650ff230b48daa19b4c199edc26f4 [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:47:44.045935 27340 raft_consensus.cc:2454] T b4e0cb30ba7a42a38c7537240ecf169f P 84d954cf7bce4867ad5f0a9f63b16b65 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8178896c2ac3402f818b46b702589893 in term 1.
I20260806 03:47:44.046051 27147 raft_consensus.cc:2454] T b4e0cb30ba7a42a38c7537240ecf169f P a0d650ff230b48daa19b4c199edc26f4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8178896c2ac3402f818b46b702589893 in term 1.
I20260806 03:47:44.046105 27444 leader_election.cc:302] T b4e0cb30ba7a42a38c7537240ecf169f P 8178896c2ac3402f818b46b702589893 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8178896c2ac3402f818b46b702589893, 84d954cf7bce4867ad5f0a9f63b16b65; no voters:
I20260806 03:47:44.046221 27631 raft_consensus.cc:2781] T b4e0cb30ba7a42a38c7537240ecf169f P 8178896c2ac3402f818b46b702589893 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 03:47:44.046406 27631 raft_consensus.cc:686] T b4e0cb30ba7a42a38c7537240ecf169f P 8178896c2ac3402f818b46b702589893 [term 1 LEADER]: Becoming Leader. State: Replica: 8178896c2ac3402f818b46b702589893, State: Running, Role: LEADER
I20260806 03:47:44.046505 27631 consensus_queue.cc:237] T b4e0cb30ba7a42a38c7537240ecf169f P 8178896c2ac3402f818b46b702589893 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8178896c2ac3402f818b46b702589893" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 45675 } } peers { permanent_uuid: "84d954cf7bce4867ad5f0a9f63b16b65" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 37893 } } peers { permanent_uuid: "a0d650ff230b48daa19b4c199edc26f4" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 36455 } }
I20260806 03:47:44.047398 26929 catalog_manager.cc:5495] T b4e0cb30ba7a42a38c7537240ecf169f P 8178896c2ac3402f818b46b702589893 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8178896c2ac3402f818b46b702589893 (127.11.253.195). New cstate: current_term: 1 leader_uuid: "8178896c2ac3402f818b46b702589893" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8178896c2ac3402f818b46b702589893" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 45675 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "84d954cf7bce4867ad5f0a9f63b16b65" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 37893 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0d650ff230b48daa19b4c199edc26f4" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 36455 } health_report { overall_health: UNKNOWN } } }
I20260806 03:47:44.065676 27618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:44.063189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35878 (local address 127.11.253.195:45675)
0806 03:47:44.063263 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0806 03:47:44.063265 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:47:44.063275 (+ 10us) server_negotiation.cc:407] Connection header received
0806 03:47:44.063326 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:47:44.063327 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:47:44.063345 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:47:44.063433 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 03:47:44.063590 (+ 157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:44.064065 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:44.064842 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:47:44.065044 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:47:44.065124 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:47:44.065310 (+ 186us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 03:47:44.065421 (+ 111us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 03:47:44.065514 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0806 03:47:44.065544 (+ 30us) server_negotiation.cc:299] Negotiation successful
0806 03:47:44.065566 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260806 03:47:44.067636 27147 raft_consensus.cc:1261] T b4e0cb30ba7a42a38c7537240ecf169f P a0d650ff230b48daa19b4c199edc26f4 [term 1 FOLLOWER]: Refusing update from remote peer 8178896c2ac3402f818b46b702589893: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 03:47:44.067633 27340 raft_consensus.cc:1261] T b4e0cb30ba7a42a38c7537240ecf169f P 84d954cf7bce4867ad5f0a9f63b16b65 [term 1 FOLLOWER]: Refusing update from remote peer 8178896c2ac3402f818b46b702589893: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 03:47:44.067991 27631 consensus_queue.cc:1035] T b4e0cb30ba7a42a38c7537240ecf169f P 8178896c2ac3402f818b46b702589893 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d954cf7bce4867ad5f0a9f63b16b65" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 37893 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 03:47:44.068178 27636 consensus_queue.cc:1035] T b4e0cb30ba7a42a38c7537240ecf169f P 8178896c2ac3402f818b46b702589893 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0d650ff230b48daa19b4c199edc26f4" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 36455 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 03:47:44.071472 27641 mvcc.cc:205] Tried to move back new op lower bound from 7315407110418079744 to 7315407110335188992. Current Snapshot: MvccSnapshot[applied={T|T < 7315407110418079744}]
I20260806 03:47:44.072249 27643 mvcc.cc:205] Tried to move back new op lower bound from 7315407110418079744 to 7315407110335188992. Current Snapshot: MvccSnapshot[applied={T|T < 7315407110418079744 or (T in {7315407110418079744})}]
I20260806 03:47:44.073387 26931 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57496:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 03:47:44.073480 26931 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57496:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 03:47:44.074316 26931 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 57eb1d8313964593a93034ad0747c1ed: Sending DeleteTablet for 3 replicas of tablet b4e0cb30ba7a42a38c7537240ecf169f
I20260806 03:47:44.074678 27499 tablet_service.cc:1514] Processing DeleteTablet for tablet b4e0cb30ba7a42a38c7537240ecf169f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 03:47:44 UTC) from {username='kudu', principal='kudu/127.11.253.254@KRBTEST.COM'} at 127.0.0.1:35876
I20260806 03:47:44.074754 27318 tablet_service.cc:1514] Processing DeleteTablet for tablet b4e0cb30ba7a42a38c7537240ecf169f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 03:47:44 UTC) from {username='kudu', principal='kudu/127.11.253.254@KRBTEST.COM'} at 127.0.0.1:45792
I20260806 03:47:44.074961 27656 tablet_replica.cc:330] T b4e0cb30ba7a42a38c7537240ecf169f P 8178896c2ac3402f818b46b702589893: stopping tablet replica
I20260806 03:47:44.074960 27120 tablet_service.cc:1514] Processing DeleteTablet for tablet b4e0cb30ba7a42a38c7537240ecf169f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 03:47:44 UTC) from {username='kudu', principal='kudu/127.11.253.254@KRBTEST.COM'} at 127.0.0.1:52696
I20260806 03:47:44.075076 27656 raft_consensus.cc:2229] T b4e0cb30ba7a42a38c7537240ecf169f P 8178896c2ac3402f818b46b702589893 [term 1 LEADER]: Raft consensus shutting down.
I20260806 03:47:44.075181 27656 raft_consensus.cc:2258] T b4e0cb30ba7a42a38c7537240ecf169f P 8178896c2ac3402f818b46b702589893 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 03:47:44.075201 12279 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskToVWRe/build/release/bin/kudu with pid 27025
I20260806 03:47:44.075730 27656 ts_tablet_manager.cc:1905] T b4e0cb30ba7a42a38c7537240ecf169f P 8178896c2ac3402f818b46b702589893: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 03:47:44.077741 27656 ts_tablet_manager.cc:1918] T b4e0cb30ba7a42a38c7537240ecf169f P 8178896c2ac3402f818b46b702589893: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 03:47:44.077811 27656 log.cc:1196] T b4e0cb30ba7a42a38c7537240ecf169f P 8178896c2ac3402f818b46b702589893: Deleting WAL directory at /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785988004637995-12279-0/minicluster-data/ts-2/wal/wals/b4e0cb30ba7a42a38c7537240ecf169f
I20260806 03:47:44.078167 27656 ts_tablet_manager.cc:1939] T b4e0cb30ba7a42a38c7537240ecf169f P 8178896c2ac3402f818b46b702589893: Deleting consensus metadata
I20260806 03:47:44.078576 26910 catalog_manager.cc:4841] TS 8178896c2ac3402f818b46b702589893 (127.11.253.195:45675): tablet b4e0cb30ba7a42a38c7537240ecf169f (table test-table [id=30fdfa43eddc46ce9e8d80a0197395e5]) successfully deleted
I20260806 03:47:44.079207 27657 tablet_replica.cc:330] T b4e0cb30ba7a42a38c7537240ecf169f P 84d954cf7bce4867ad5f0a9f63b16b65: stopping tablet replica
I20260806 03:47:44.079397 27657 raft_consensus.cc:2229] T b4e0cb30ba7a42a38c7537240ecf169f P 84d954cf7bce4867ad5f0a9f63b16b65 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 03:47:44.079558 27657 raft_consensus.cc:2258] T b4e0cb30ba7a42a38c7537240ecf169f P 84d954cf7bce4867ad5f0a9f63b16b65 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 03:47:44.081027 27657 ts_tablet_manager.cc:1905] T b4e0cb30ba7a42a38c7537240ecf169f P 84d954cf7bce4867ad5f0a9f63b16b65: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 03:47:44.082787 27657 ts_tablet_manager.cc:1918] T b4e0cb30ba7a42a38c7537240ecf169f P 84d954cf7bce4867ad5f0a9f63b16b65: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 03:47:44.082953 27657 log.cc:1196] T b4e0cb30ba7a42a38c7537240ecf169f P 84d954cf7bce4867ad5f0a9f63b16b65: Deleting WAL directory at /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785988004637995-12279-0/minicluster-data/ts-1/wal/wals/b4e0cb30ba7a42a38c7537240ecf169f
I20260806 03:47:44.083310 27657 ts_tablet_manager.cc:1939] T b4e0cb30ba7a42a38c7537240ecf169f P 84d954cf7bce4867ad5f0a9f63b16b65: Deleting consensus metadata
I20260806 03:47:44.083799 26913 catalog_manager.cc:4841] TS 84d954cf7bce4867ad5f0a9f63b16b65 (127.11.253.194:37893): tablet b4e0cb30ba7a42a38c7537240ecf169f (table test-table [id=30fdfa43eddc46ce9e8d80a0197395e5]) successfully deleted
W20260806 03:47:44.085476 26912 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.253.193:36455 (error 108)
I20260806 03:47:44.086103 27614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:47:44.085955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.193:36455 (local address 127.0.0.1:52704)
0806 03:47:44.086015 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0806 03:47:44.086059 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.253.193:36455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260806 03:47:44.086202 12279 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskToVWRe/build/release/bin/kudu with pid 27222
W20260806 03:47:44.086279 26912 catalog_manager.cc:4584] TS a0d650ff230b48daa19b4c199edc26f4 (127.11.253.193:36455): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b4e0cb30ba7a42a38c7537240ecf169f: Network error: Client connection negotiation failed: client connection to 127.11.253.193:36455: connect: Connection refused (error 111)
I20260806 03:47:44.101703 12279 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskToVWRe/build/release/bin/kudu with pid 27412
I20260806 03:47:44.115159 12279 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskToVWRe/build/release/bin/kudu with pid 26883
2026-08-06T03:47:44Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (2318 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988004637995-12279-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 03:47:44 dist-test-slave-6ntk krb5kdc[27666](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 03:47:44 dist-test-slave-6ntk krb5kdc[27666](info): set up 2 sockets
Aug 06 03:47:44 dist-test-slave-6ntk krb5kdc[27666](info): commencing operation
krb5kdc: starting...
W20260806 03:47:45.335600 12279 mini_kdc.cc:121] Time spent starting KDC: real 1.192s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988004637995-12279-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988004637995-12279-0/krb5kdc/test-user.keytab.
Aug 06 03:47:45 dist-test-slave-6ntk krb5kdc[27666](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988065, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T03:47:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T03:47:45Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.11.253.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.11.253.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.11.253.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988004637995-12279-0/krb5kdc/rangeradmin_127.11.253.212@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.11.253.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988004637995-12279-0/krb5kdc/rangeradmin_127.11.253.212@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.11.253.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.11.253.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.11.253.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988004637995-12279-0/krb5kdc/rangerlookup_127.11.253.212@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.11.253.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988004637995-12279-0/krb5kdc/rangerlookup_127.11.253.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.11.253.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.11.253.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.11.253.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988004637995-12279-0/krb5kdc/HTTP_127.11.253.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.11.253.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988004637995-12279-0/krb5kdc/HTTP_127.11.253.212@KRBTEST.COM.keytab.
I20260806 03:47:45.517205 12279 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988004637995-12279-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskToVWRe/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988004637995-12279-0/minicluster-data/postgres -l logfile start
2026-08-06 03:47:48.178 UTC [27710] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-06 03:47:48.178 UTC [27710] LOG: listening on IPv4 address "127.11.253.212", port 35847
2026-08-06 03:47:48.183 UTC [27710] LOG: listening on Unix socket "/tmp/.s.PGSQL.35847"
2026-08-06 03:47:48.215 UTC [27714] LOG: database system was shut down at 2026-08-06 03:47:46 UTC
2026-08-06 03:47:48.220 UTC [27710] LOG: database system is ready to accept connections
I20260806 03:47:49.336453 12279 mini_postgres.cc:96] Postgres bound to 35847
2026-08-06 03:47:49.344 UTC [27727] FATAL: database "slave" does not exist
127.11.253.212:35847 - accepting connections
I20260806 03:47:49.344830 12279 mini_ranger.cc:161] Starting Ranger...
I20260806 03:47:49.361171 12279 mini_ranger.cc:84] Created miniranger Postgres user
I20260806 03:47:49.880105 12279 mini_ranger.cc:87] Created ranger Postgres database
I20260806 03:47:49.880204 12279 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988004637995-12279-0/minicluster-data/ranger-admin
2026-08-06 03:47:50,265 [I] DB FLAVOR :POSTGRES
2026-08-06 03:47:50,266 [I] --------- Verifying Ranger DB connection ---------
2026-08-06 03:47:50,266 [I] Checking connection..
2026-08-06 03:47:50,266 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskToVWRe/build/release/bin/postgresql.jar:/tmp/dist-test-taskToVWRe/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.253.212:35847/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-06 03:47:50,549 [I] Checking connection passed.
2026-08-06 03:47:50,549 [I] --------- Verifying version history table ---------
2026-08-06 03:47:50,549 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskToVWRe/build/release/bin/postgresql.jar:/tmp/dist-test-taskToVWRe/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.253.212:35847/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 03:47:50,859 [I] Table x_db_version_h does not exist in database ranger
2026-08-06 03:47:50,859 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskToVWRe/build/release/bin/postgresql.jar:/tmp/dist-test-taskToVWRe/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.253.212:35847/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 03:47:51,118 [I] Table x_db_version_h does not exist in database ranger
2026-08-06 03:47:51,118 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-06 03:47:51,118 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskToVWRe/build/release/bin/postgresql.jar:/tmp/dist-test-taskToVWRe/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.253.212:35847/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskToVWRe/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-06 03:47:51.387 UTC [27814] WARNING: there is no transaction in progress
2026-08-06 03:47:51,396 [I] create_dbversion_catalog.sql file imported successfully
2026-08-06 03:47:51,396 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskToVWRe/build/release/bin/postgresql.jar:/tmp/dist-test-taskToVWRe/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.253.212:35847/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 03:47:51,695 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-06 03:47:51,695 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-06 03:47:51,696 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskToVWRe/build/release/bin/postgresql.jar:/tmp/dist-test-taskToVWRe/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.253.212:35847/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-06 03:47:51,977 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskToVWRe/build/release/bin/postgresql.jar:/tmp/dist-test-taskToVWRe/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.253.212:35847/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-06 03:47:52,234 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskToVWRe/build/release/bin/postgresql.jar:/tmp/dist-test-taskToVWRe/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.253.212:35847/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-6ntk.c.gcp-upstream.internal','N') ;"
2026-08-06 03:47:52,502 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskToVWRe/build/release/bin/postgresql.jar:/tmp/dist-test-taskToVWRe/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.253.212:35847/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-06 03:47:52,757 [I] Table x_portal_user does not exist in database ranger
2026-08-06 03:47:52,757 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskToVWRe/build/release/bin/postgresql.jar:/tmp/dist-test-taskToVWRe/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.253.212:35847/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-06 03:47:53,025 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-06 03:47:53,025 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskToVWRe/build/release/bin/postgresql.jar:/tmp/dist-test-taskToVWRe/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.253.212:35847/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-06 03:47:53,280 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskToVWRe/build/release/bin/postgresql.jar:/tmp/dist-test-taskToVWRe/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.253.212:35847/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-06 03:47:53,528 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-06 03:47:53,528 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskToVWRe/build/release/bin/postgresql.jar:/tmp/dist-test-taskToVWRe/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.253.212:35847/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskToVWRe/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-06 03:47:54.566 UTC [27976] WARNING: there is no transaction in progress
2026-08-06 03:47:54.584 UTC [27976] WARNING: there is no transaction in progress
2026-08-06 03:47:54.603 UTC [27976] WARNING: there is no transaction in progress
2026-08-06 03:47:54.620 UTC [27976] WARNING: there is no transaction in progress
2026-08-06 03:47:54.638 UTC [27976] WARNING: there is no transaction in progress
2026-08-06 03:47:54.660 UTC [27976] WARNING: there is no transaction in progress
2026-08-06 03:47:54.733 UTC [27976] WARNING: there is no transaction in progress
2026-08-06 03:47:54.741 UTC [27976] WARNING: there is no transaction in progress
2026-08-06 03:47:54.757 UTC [27976] WARNING: there is no transaction in progress
2026-08-06 03:47:54.767 UTC [27976] WARNING: there is no transaction in progress
2026-08-06 03:47:54.778 UTC [27976] WARNING: there is no transaction in progress
2026-08-06 03:47:54.793 UTC [27976] WARNING: there is no transaction in progress
2026-08-06 03:47:54.803 UTC [27976] WARNING: there is no transaction in progress
2026-08-06 03:47:54.810 UTC [27976] WARNING: there is no transaction in progress
2026-08-06 03:47:54.818 UTC [27976] WARNING: there is no transaction in progress
2026-08-06 03:47:54.824 UTC [27976] WARNING: there is no transaction in progress
2026-08-06 03:47:55,526 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-06 03:47:55,526 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskToVWRe/build/release/bin/postgresql.jar:/tmp/dist-test-taskToVWRe/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.253.212:35847/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-06 03:47:55,782 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-06 03:47:55,783 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskToVWRe/build/release/bin/postgresql.jar:/tmp/dist-test-taskToVWRe/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.253.212:35847/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-06 03:47:56,020 [I] Table x_portal_user already exists in database 'ranger'
2026-08-06 03:47:56,020 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskToVWRe/build/release/bin/postgresql.jar:/tmp/dist-test-taskToVWRe/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.253.212:35847/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-06 03:47:56,277 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-06 03:47:56,278 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskToVWRe/build/release/bin/postgresql.jar:/tmp/dist-test-taskToVWRe/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.253.212:35847/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-06 03:47:56,541 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskToVWRe/build/release/bin/postgresql.jar:/tmp/dist-test-taskToVWRe/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.253.212:35847/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-06 03:47:56,798 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskToVWRe/build/release/bin/postgresql.jar:/tmp/dist-test-taskToVWRe/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.253.212:35847/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-6ntk.c.gcp-upstream.internal';"
2026-08-06 03:47:57,046 [I] CORE_DB_SCHEMA import status has been updated
2026-08-06 03:47:57,046 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskToVWRe/build/release/bin/postgresql.jar:/tmp/dist-test-taskToVWRe/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.253.212:35847/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-06 03:47:57,305 [I] DB_PATCHES have already been applied
I20260806 03:47:57.311671 12279 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988004637995-12279-0/minicluster-data/ranger-admin:/tmp/dist-test-taskToVWRe/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskToVWRe/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskToVWRe/build/release/bin/postgresql.jar:/tmp/dist-test-taskToVWRe/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260806 03:47:57.311741 12279 mini_ranger.cc:193] Using host: 127.11.253.212
I20260806 03:47:57.315577 12279 mini_ranger.cc:227] Ranger admin URL: http://127.11.253.212:35255
Aug 06, 2026 3:47:57 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 06, 2026 3:47:57 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 06, 2026 3:47:57 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988004637995-12279-0/minicluster-data/ranger-admin/ews/webapp
Aug 06, 2026 3:47:57 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988004637995-12279-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 06, 2026 3:47:57 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988004637995-12279-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 06, 2026 3:47:58 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988004637995-12279-0/minicluster-data/ranger-admin/ews/webapp].
Aug 06, 2026 3:47:58 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260806 03:47:58.738582 12279 mini_ranger.cc:160] Time spent starting Ranger: real 9.394s user 0.000s sys 0.011s
Aug 06, 2026 3:48:20 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 06, 2026 3:48:20 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Aug 06, 2026 3:48:20 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 06, 2026 3:48:20 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 06, 2026 3:48:20 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 06, 2026 3:48:20 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 06, 2026 3:48:20 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 06, 2026 3:48:20 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 06, 2026 3:48:20 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 06, 2026 3:48:20 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 06, 2026 3:48:20 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 06, 2026 3:48:20 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 06, 2026 3:48:20 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 06, 2026 3:48:20 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 06, 2026 3:48:20 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 06, 2026 3:48:20 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 06, 2026 3:48:20 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 06, 2026 3:48:21 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260806 03:48:22.196026 12279 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.11.253.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.11.253.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.11.253.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988004637995-12279-0/krb5kdc/rangerkms_127.11.253.212@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.11.253.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988004637995-12279-0/krb5kdc/rangerkms_127.11.253.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.11.253.212@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.11.253.212@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.11.253.212@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988004637995-12279-0/krb5kdc/HTTP_127.11.253.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.11.253.212@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988004637995-12279-0/krb5kdc/HTTP_127.11.253.212@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 06 03:48:22 dist-test-slave-6ntk krb5kdc[27666](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988102, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260806 03:48:22.296540 12279 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260806 03:48:22.308744 12279 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260806 03:48:23.302721 12279 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260806 03:48:23.302811 12279 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988004637995-12279-0/minicluster-data/ranger-kms
I20260806 03:48:23.302835 12279 mini_ranger_kms.cc:227] Using postgres at 127.11.253.212:35847
2026-08-06 03:48:23,540 [I] DB FLAVOR :POSTGRES
2026-08-06 03:48:23,540 [I] --------- Verifying Ranger DB connection ---------
2026-08-06 03:48:23,540 [I] Checking connection
2026-08-06 03:48:23,540 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskToVWRe/build/release/bin/postgresql.jar:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988004637995-12279-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.253.212:35847/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-06 03:48:23,801 [I] connection success
2026-08-06 03:48:23,802 [I] --------- Verifying Ranger DB tables ---------
2026-08-06 03:48:23,802 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-06 03:48:23,802 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskToVWRe/build/release/bin/postgresql.jar:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988004637995-12279-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.253.212:35847/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-06 03:48:24,084 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-06 03:48:24,084 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-06 03:48:24,084 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-06 03:48:24,085 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskToVWRe/build/release/bin/postgresql.jar:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988004637995-12279-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.253.212:35847/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988004637995-12279-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-06 03:48:24,380 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260806 03:48:24.901917 12279 mini_ranger_kms.cc:310] Created kms service
I20260806 03:48:25.147943 12279 mini_ranger_kms.cc:326] Created kudu user
I20260806 03:48:25.194380 12279 mini_ranger_kms.cc:343] Created rangerkms user
Aug 06 03:48:25 dist-test-slave-6ntk krb5kdc[27666](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988102, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.11.253.212@KRBTEST.COM
I20260806 03:48:25.630569 12279 mini_ranger_kms.cc:384] Added ranger policy
I20260806 03:48:25.630702 12279 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988004637995-12279-0/minicluster-data/ranger-kms:/tmp/dist-test-taskToVWRe/build/release/bin/postgresql.jar:/tmp/dist-test-taskToVWRe/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskToVWRe/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260806 03:48:25.630725 12279 mini_ranger_kms.cc:240] Using host: 127.11.253.212
I20260806 03:48:25.635079 12279 mini_ranger_kms.cc:276] Ranger KMS PID: 28251
I20260806 03:48:25.635239 12279 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.11.253.212:55111
Aug 06, 2026 3:48:26 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 06, 2026 3:48:26 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 06, 2026 3:48:27 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988004637995-12279-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 06, 2026 3:48:27 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988004637995-12279-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 06, 2026 3:48:27 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988004637995-12279-0/minicluster-data/ranger-kms/ews/webapp].
Aug 06, 2026 3:48:27 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260806 03:48:28.278072 12279 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.982s user 0.005s sys 0.213s
I20260806 03:48:28.278200 12279 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260806 03:48:28.278301 12279 mini_ranger_kms.cc:401] 127.11.253.212:55111/kms/v1/keys
Aug 06 03:48:28 dist-test-slave-6ntk krb5kdc[27666](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1785988108, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.11.253.212@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 06 03:48:28 dist-test-slave-6ntk krb5kdc[27666](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988108, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.11.253.212@KRBTEST.COM for HTTP/127.11.253.212@KRBTEST.COM
Aug 06 03:48:29 dist-test-slave-6ntk krb5kdc[27666](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988108, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.11.253.212@KRBTEST.COM for HTTP/127.11.253.212@KRBTEST.COM
Aug 06 03:48:31 dist-test-slave-6ntk krb5kdc[27666](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988111, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.11.253.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.253.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.253.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.254.keytab.
Entry for principal kudu/127.11.253.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.254.keytab.
I20260806 03:48:31.525756 12279 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskToVWRe/build/release/bin/kudu
/tmp/dist-test-taskToVWRe/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988004637995-12279-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988004637995-12279-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988004637995-12279-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988004637995-12279-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.253.254:39491
--webserver_interface=127.11.253.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.254.keytab
--principal=kudu/127.11.253.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.253.212:46027
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.253.254:39491
--ranger_config_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988004637995-12279-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.11.253.212:55111/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988004637995-12279-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988004637995-12279-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988004637995-12279-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:48:31.637576 28311 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:48:31.637805 28311 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:48:31.637833 28311 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:48:31.638974 28311 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 03:48:31.639012 28311 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:48:31.639026 28311 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:48:31.639038 28311 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 03:48:31.639050 28311 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 03:48:31.640726 28311 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.253.212:46027
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988004637995-12279-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988004637995-12279-0/minicluster-data/master-0/wal
--ranger_kms_url=127.11.253.212:55111/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.11.253.254:39491
--ranger_config_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988004637995-12279-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.254.keytab
--principal=kudu/127.11.253.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.253.254:39491
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988004637995-12279-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.253.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28311
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988004637995-12279-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:48:31.640938 28311 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:48:31.641119 28311 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:48:31.646803 28321 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:48:31.646816 28323 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:48:31.646847 28311 server_base.cc:1034] running on GCE node
W20260806 03:48:31.646812 28318 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:48:31.647305 28311 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:48:31.647634 28311 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:48:31.648836 28311 hybrid_clock.cc:648] HybridClock initialized: now 1785988111648817 us; error 66 us; skew 500 ppm
Aug 06 03:48:31 dist-test-slave-6ntk krb5kdc[27666](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988111, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.253.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:48:31.655490 28311 init.cc:377] Logged in from keytab as kudu/127.11.253.254@KRBTEST.COM (short username kudu)
I20260806 03:48:31.656066 28311 webserver.cc:460] Webserver started at http://127.11.253.254:39147/ using document root <none> and password file <none>
I20260806 03:48:31.656335 28311 fs_manager.cc:359] Metadata directory not provided
I20260806 03:48:31.656400 28311 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:48:31.656562 28311 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 03:48:31 dist-test-slave-6ntk krb5kdc[27666](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988111, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.253.254@KRBTEST.COM for HTTP/127.11.253.212@KRBTEST.COM
I20260806 03:48:31.724224 28311 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988004637995-12279-0/minicluster-data/master-0/data/instance:
uuid: "8f31c12182a5454db3f2321acd89cf00"
format_stamp: "Formatted at 2026-08-06 03:48:31 on dist-test-slave-6ntk"
server_key: "0e2966bf87a71ded83e452fbd1998d41"
server_key_iv: "abbd78a3d6fe645aaa3fec46d97d4047"
server_key_version: "kuduclusterkey@0"
I20260806 03:48:31.724737 28311 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988004637995-12279-0/minicluster-data/master-0/wal/instance:
uuid: "8f31c12182a5454db3f2321acd89cf00"
format_stamp: "Formatted at 2026-08-06 03:48:31 on dist-test-slave-6ntk"
server_key: "0e2966bf87a71ded83e452fbd1998d41"
server_key_iv: "abbd78a3d6fe645aaa3fec46d97d4047"
server_key_version: "kuduclusterkey@0"
I20260806 03:48:31.726960 28311 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260806 03:48:31.744062 28331 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:48:31.744451 28311 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 03:48:31.744591 28311 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988004637995-12279-0/minicluster-data/master-0/data,/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988004637995-12279-0/minicluster-data/master-0/wal
uuid: "8f31c12182a5454db3f2321acd89cf00"
format_stamp: "Formatted at 2026-08-06 03:48:31 on dist-test-slave-6ntk"
server_key: "0e2966bf87a71ded83e452fbd1998d41"
server_key_iv: "abbd78a3d6fe645aaa3fec46d97d4047"
server_key_version: "kuduclusterkey@0"
I20260806 03:48:31.744715 28311 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988004637995-12279-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988004637995-12279-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988004637995-12279-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:48:31.782231 28311 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:48:31.784744 28311 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:48:31.784937 28311 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:48:31.790788 28311 rpc_server.cc:307] RPC server started. Bound to: 127.11.253.254:39491
I20260806 03:48:31.790910 28407 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.253.254:39491 every 8 connection(s)
I20260806 03:48:31.791267 28311 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988004637995-12279-0/minicluster-data/master-0/data/info.pb
I20260806 03:48:31.792048 28408 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:48:31.794627 28408 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8f31c12182a5454db3f2321acd89cf00: Bootstrap starting.
I20260806 03:48:31.795254 12279 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskToVWRe/build/release/bin/kudu as pid 28311
I20260806 03:48:31.795308 28408 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8f31c12182a5454db3f2321acd89cf00: Neither blocks nor log segments found. Creating new log.
I20260806 03:48:31.795396 12279 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988004637995-12279-0/minicluster-data/master-0/wal/instance
I20260806 03:48:31.795591 12279 external_mini_cluster.cc:1428] Setting key 24034c95ad8d37c7a9ce78d1fbb3a76b
I20260806 03:48:31.795641 28408 log.cc:824] T 00000000000000000000000000000000 P 8f31c12182a5454db3f2321acd89cf00: Log is configured to *not* fsync() on all Append() calls
I20260806 03:48:31.796561 28408 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8f31c12182a5454db3f2321acd89cf00: No bootstrap required, opened a new log
I20260806 03:48:31.799213 28408 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8f31c12182a5454db3f2321acd89cf00 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f31c12182a5454db3f2321acd89cf00" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 39491 } }
I20260806 03:48:31.799402 28408 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8f31c12182a5454db3f2321acd89cf00 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:48:31.799438 28408 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8f31c12182a5454db3f2321acd89cf00 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f31c12182a5454db3f2321acd89cf00, State: Initialized, Role: FOLLOWER
I20260806 03:48:31.799559 28408 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8f31c12182a5454db3f2321acd89cf00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f31c12182a5454db3f2321acd89cf00" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 39491 } }
I20260806 03:48:31.799633 28408 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8f31c12182a5454db3f2321acd89cf00 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 03:48:31.799662 28408 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8f31c12182a5454db3f2321acd89cf00 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 03:48:31.799700 28408 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8f31c12182a5454db3f2321acd89cf00 [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:48:31.800515 28408 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8f31c12182a5454db3f2321acd89cf00 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f31c12182a5454db3f2321acd89cf00" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 39491 } }
I20260806 03:48:31.800617 28408 leader_election.cc:302] T 00000000000000000000000000000000 P 8f31c12182a5454db3f2321acd89cf00 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8f31c12182a5454db3f2321acd89cf00; no voters:
I20260806 03:48:31.800822 28408 leader_election.cc:288] T 00000000000000000000000000000000 P 8f31c12182a5454db3f2321acd89cf00 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 03:48:31.800920 28414 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8f31c12182a5454db3f2321acd89cf00 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 03:48:31.801044 28414 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8f31c12182a5454db3f2321acd89cf00 [term 1 LEADER]: Becoming Leader. State: Replica: 8f31c12182a5454db3f2321acd89cf00, State: Running, Role: LEADER
I20260806 03:48:31.801134 28408 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8f31c12182a5454db3f2321acd89cf00 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 03:48:31.801805 28414 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8f31c12182a5454db3f2321acd89cf00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f31c12182a5454db3f2321acd89cf00" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 39491 } }
I20260806 03:48:31.802212 28414 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8f31c12182a5454db3f2321acd89cf00 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8f31c12182a5454db3f2321acd89cf00. Latest consensus state: current_term: 1 leader_uuid: "8f31c12182a5454db3f2321acd89cf00" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f31c12182a5454db3f2321acd89cf00" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 39491 } } }
I20260806 03:48:31.802294 28414 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8f31c12182a5454db3f2321acd89cf00 [sys.catalog]: This master's current role is: LEADER
I20260806 03:48:31.802531 28422 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 03:48:31.802712 28415 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8f31c12182a5454db3f2321acd89cf00 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8f31c12182a5454db3f2321acd89cf00" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f31c12182a5454db3f2321acd89cf00" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 39491 } } }
I20260806 03:48:31.802801 28415 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8f31c12182a5454db3f2321acd89cf00 [sys.catalog]: This master's current role is: LEADER
I20260806 03:48:31.803296 28422 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 03:48:31.803828 28408 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988004637995-12279-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
Aug 06 03:48:31 dist-test-slave-6ntk krb5kdc[27666](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988111, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.253.254@KRBTEST.COM
I20260806 03:48:31.812435 28412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:48:31.796201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47438 (local address 127.11.253.254:39491)
0806 03:48:31.796390 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0806 03:48:31.796400 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 03:48:31.796433 (+ 33us) server_negotiation.cc:407] Connection header received
0806 03:48:31.797088 (+ 655us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:48:31.797097 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:48:31.797170 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:48:31.797334 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:48:31.798070 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:48:31.800073 (+ 2003us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:48:31.801108 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:48:31.801556 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:48:31.801642 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:48:31.806540 (+ 4898us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:48:31.806593 (+ 53us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:48:31.806593 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:48:31.806670 (+ 77us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:48:31.810632 (+ 3962us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:48:31.811058 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:48:31.811061 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:48:31.811062 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:48:31.811152 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:48:31.811382 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:48:31.811383 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:48:31.811383 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:48:31.811654 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:48:31.811840 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0806 03:48:31.811921 (+ 81us) server_negotiation.cc:299] Negotiation successful
0806 03:48:31.812074 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":81,"threads_started":1}
I20260806 03:48:31.813282 28422 catalog_manager.cc:1347] Generated new cluster ID: d6d8a8402da6440f98fbbe36e7468a7a
I20260806 03:48:31.813351 28422 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 03:48:31.834271 28422 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 03:48:31.835651 28422 catalog_manager.cc:1504] Loading token signing keys...
I20260806 03:48:31.850790 28422 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8f31c12182a5454db3f2321acd89cf00: Generated new TSK 0
I20260806 03:48:31.851102 28422 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.253.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.253.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.253.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.193.keytab.
Entry for principal kudu/127.11.253.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.193.keytab.
I20260806 03:48:32.702045 12279 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskToVWRe/build/release/bin/kudu
/tmp/dist-test-taskToVWRe/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988004637995-12279-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988004637995-12279-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988004637995-12279-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988004637995-12279-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.11.253.212:55111/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.253.193:0
--local_ip_for_outbound_sockets=127.11.253.193
--webserver_interface=127.11.253.193
--webserver_port=0
--tserver_master_addrs=127.11.253.254:39491
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.193.keytab
--principal=kudu/127.11.253.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.253.212:46027
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988004637995-12279-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988004637995-12279-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988004637995-12279-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:48:32.826885 28474 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:48:32.827159 28474 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:48:32.827201 28474 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:48:32.828855 28474 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:48:32.828924 28474 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:48:32.828966 28474 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.253.193
I20260806 03:48:32.831351 28474 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.253.212:46027
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988004637995-12279-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988004637995-12279-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.11.253.212:55111/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.193.keytab
--principal=kudu/127.11.253.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.253.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988004637995-12279-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.253.193
--webserver_port=0
--tserver_master_addrs=127.11.253.254:39491
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28474
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.253.193
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988004637995-12279-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:48:32.831640 28474 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:48:32.831920 28474 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:48:32.839043 28486 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:48:32.839051 28485 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:48:32.839236 28474 server_base.cc:1034] running on GCE node
W20260806 03:48:32.839474 28488 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:48:32.839740 28474 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:48:32.840057 28474 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:48:32.841246 28474 hybrid_clock.cc:648] HybridClock initialized: now 1785988112841242 us; error 77 us; skew 500 ppm
Aug 06 03:48:32 dist-test-slave-6ntk krb5kdc[27666](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988112, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.253.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:48:32.844944 28474 init.cc:377] Logged in from keytab as kudu/127.11.253.193@KRBTEST.COM (short username kudu)
I20260806 03:48:32.845520 28474 webserver.cc:460] Webserver started at http://127.11.253.193:42565/ using document root <none> and password file <none>
I20260806 03:48:32.845779 28474 fs_manager.cc:359] Metadata directory not provided
I20260806 03:48:32.845844 28474 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:48:32.846020 28474 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 03:48:32 dist-test-slave-6ntk krb5kdc[27666](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988112, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.253.193@KRBTEST.COM for HTTP/127.11.253.212@KRBTEST.COM
I20260806 03:48:32.860514 28474 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988004637995-12279-0/minicluster-data/ts-0/data/instance:
uuid: "87f8ddf1bae0432d844ce4b79fcb15ff"
format_stamp: "Formatted at 2026-08-06 03:48:32 on dist-test-slave-6ntk"
server_key: "e296b5107ffb653d5b3b3ea89de39eb7"
server_key_iv: "e07387894653dc7a5b32ad8696e9e929"
server_key_version: "kuduclusterkey@0"
I20260806 03:48:32.860971 28474 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988004637995-12279-0/minicluster-data/ts-0/wal/instance:
uuid: "87f8ddf1bae0432d844ce4b79fcb15ff"
format_stamp: "Formatted at 2026-08-06 03:48:32 on dist-test-slave-6ntk"
server_key: "e296b5107ffb653d5b3b3ea89de39eb7"
server_key_iv: "e07387894653dc7a5b32ad8696e9e929"
server_key_version: "kuduclusterkey@0"
I20260806 03:48:32.862829 28474 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.003s
I20260806 03:48:32.878690 28500 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:48:32.878914 28474 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 03:48:32.879055 28474 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988004637995-12279-0/minicluster-data/ts-0/data,/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988004637995-12279-0/minicluster-data/ts-0/wal
uuid: "87f8ddf1bae0432d844ce4b79fcb15ff"
format_stamp: "Formatted at 2026-08-06 03:48:32 on dist-test-slave-6ntk"
server_key: "e296b5107ffb653d5b3b3ea89de39eb7"
server_key_iv: "e07387894653dc7a5b32ad8696e9e929"
server_key_version: "kuduclusterkey@0"
I20260806 03:48:32.879172 28474 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988004637995-12279-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988004637995-12279-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988004637995-12279-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:48:32.898090 28474 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:48:32.900556 28474 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:48:32.900709 28474 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:48:32.901000 28474 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 03:48:32.901387 28474 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 03:48:32.901439 28474 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 03:48:32.901479 28474 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 03:48:32.901500 28474 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 03:48:32.911123 28474 rpc_server.cc:307] RPC server started. Bound to: 127.11.253.193:41291
I20260806 03:48:32.911585 28474 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988004637995-12279-0/minicluster-data/ts-0/data/info.pb
I20260806 03:48:32.912581 28659 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.253.193:41291 every 8 connection(s)
I20260806 03:48:32.920001 12279 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskToVWRe/build/release/bin/kudu as pid 28474
I20260806 03:48:32.920135 12279 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988004637995-12279-0/minicluster-data/ts-0/wal/instance
I20260806 03:48:32.920357 12279 external_mini_cluster.cc:1428] Setting key c8bc9f3a55d14f1771111482b7c9b49d
Aug 06 03:48:32 dist-test-slave-6ntk krb5kdc[27666](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988112, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.253.193@KRBTEST.COM for kudu/127.11.253.254@KRBTEST.COM
I20260806 03:48:32.925185 28662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:48:32.913434 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.253.193:36645 (local address 127.11.253.254:39491)
0806 03:48:32.913605 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0806 03:48:32.913608 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 03:48:32.913851 (+ 243us) server_negotiation.cc:407] Connection header received
0806 03:48:32.914634 (+ 783us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:48:32.914636 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:48:32.914653 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:48:32.914696 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:48:32.915599 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:48:32.916387 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:48:32.917547 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:48:32.917768 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:48:32.917853 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:48:32.921019 (+ 3166us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:48:32.921021 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:48:32.921021 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:48:32.921029 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:48:32.923103 (+ 2074us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:48:32.923753 (+ 650us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:48:32.923755 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:48:32.923755 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:48:32.923883 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:48:32.924285 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:48:32.924287 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:48:32.924288 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:48:32.924464 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:48:32.924540 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0806 03:48:32.924920 (+ 380us) server_negotiation.cc:299] Negotiation successful
0806 03:48:32.925024 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":79,"threads_started":1}
I20260806 03:48:32.925311 28665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:48:32.913425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:39491 (local address 127.11.253.193:36645)
0806 03:48:32.913703 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0806 03:48:32.913747 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0806 03:48:32.914492 (+ 745us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:48:32.914904 (+ 412us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:48:32.914909 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:48:32.915186 (+ 277us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:48:32.915480 (+ 294us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:48:32.915487 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:48:32.916499 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:48:32.916500 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:48:32.917438 (+ 938us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:48:32.917440 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:48:32.917908 (+ 468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:48:32.917909 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:48:32.918134 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:48:32.919006 (+ 872us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:48:32.919018 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:48:32.920915 (+ 1897us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:48:32.923227 (+ 2312us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:48:32.923235 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:48:32.923238 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:48:32.923658 (+ 420us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:48:32.923976 (+ 318us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:48:32.923977 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:48:32.923977 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:48:32.924169 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:48:32.924548 (+ 379us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:48:32.924555 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:48:32.924731 (+ 176us) client_negotiation.cc:769] Sending connection context
0806 03:48:32.924878 (+ 147us) client_negotiation.cc:240] Negotiation successful
0806 03:48:32.925065 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":235,"thread_start_us":198,"threads_started":1}
I20260806 03:48:32.926380 28660 heartbeater.cc:347] Connected to a master server at 127.11.253.254:39491
I20260806 03:48:32.926506 28660 heartbeater.cc:464] Registering TS with master...
I20260806 03:48:32.926709 28660 heartbeater.cc:511] Master 127.11.253.254:39491 requested a full tablet report, sending...
I20260806 03:48:32.927386 28357 ts_manager.cc:194] Registered new tserver with Master: 87f8ddf1bae0432d844ce4b79fcb15ff (127.11.253.193:41291)
I20260806 03:48:32.928378 28357 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.253.193@KRBTEST.COM'} at 127.11.253.193:36645
WARNING: no policy specified for kudu/127.11.253.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.253.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.253.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.194.keytab.
Entry for principal kudu/127.11.253.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.194.keytab.
I20260806 03:48:32.954797 12279 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskToVWRe/build/release/bin/kudu
/tmp/dist-test-taskToVWRe/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988004637995-12279-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988004637995-12279-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988004637995-12279-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988004637995-12279-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.11.253.212:55111/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.253.194:0
--local_ip_for_outbound_sockets=127.11.253.194
--webserver_interface=127.11.253.194
--webserver_port=0
--tserver_master_addrs=127.11.253.254:39491
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.194.keytab
--principal=kudu/127.11.253.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.253.212:46027
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988004637995-12279-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988004637995-12279-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988004637995-12279-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:48:33.091956 28668 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:48:33.092257 28668 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:48:33.092294 28668 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:48:33.094101 28668 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:48:33.094161 28668 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:48:33.094201 28668 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.253.194
Aug 06 03:48:33 dist-test-slave-6ntk krb5kdc[27666](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1785988113, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.253.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:48:33.096679 28668 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.253.212:46027
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988004637995-12279-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988004637995-12279-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.11.253.212:55111/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.194.keytab
--principal=kudu/127.11.253.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.253.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988004637995-12279-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.253.194
--webserver_port=0
--tserver_master_addrs=127.11.253.254:39491
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28668
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.253.194
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988004637995-12279-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:48:33.096949 28668 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:48:33.097239 28668 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:48:33.106053 28679 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:48:33.106387 28684 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:48:33.106604 28680 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:48:33.110673 28668 server_base.cc:1034] running on GCE node
I20260806 03:48:33.110961 28668 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:48:33.111279 28668 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:48:33.114591 28668 hybrid_clock.cc:648] HybridClock initialized: now 1785988113114560 us; error 44 us; skew 500 ppm
Aug 06 03:48:33 dist-test-slave-6ntk krb5kdc[27666](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988113, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.253.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:48:33.118382 28668 init.cc:377] Logged in from keytab as kudu/127.11.253.194@KRBTEST.COM (short username kudu)
I20260806 03:48:33.118964 28668 webserver.cc:460] Webserver started at http://127.11.253.194:36885/ using document root <none> and password file <none>
I20260806 03:48:33.119238 28668 fs_manager.cc:359] Metadata directory not provided
I20260806 03:48:33.119304 28668 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:48:33.119451 28668 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 03:48:33 dist-test-slave-6ntk krb5kdc[27666](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988113, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.253.194@KRBTEST.COM for HTTP/127.11.253.212@KRBTEST.COM
I20260806 03:48:33.135205 28668 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988004637995-12279-0/minicluster-data/ts-1/data/instance:
uuid: "0b44f08250a042d1a6105506d57e4c6a"
format_stamp: "Formatted at 2026-08-06 03:48:33 on dist-test-slave-6ntk"
server_key: "7aaba10cc4faaf09c27942a205bc7d50"
server_key_iv: "ada5bad631e5520463d02762c94d7bd6"
server_key_version: "kuduclusterkey@0"
I20260806 03:48:33.135836 28668 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988004637995-12279-0/minicluster-data/ts-1/wal/instance:
uuid: "0b44f08250a042d1a6105506d57e4c6a"
format_stamp: "Formatted at 2026-08-06 03:48:33 on dist-test-slave-6ntk"
server_key: "7aaba10cc4faaf09c27942a205bc7d50"
server_key_iv: "ada5bad631e5520463d02762c94d7bd6"
server_key_version: "kuduclusterkey@0"
I20260806 03:48:33.142467 28668 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.000s sys 0.002s
I20260806 03:48:33.162761 28694 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:48:33.163138 28668 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.000s sys 0.001s
I20260806 03:48:33.163287 28668 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988004637995-12279-0/minicluster-data/ts-1/data,/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988004637995-12279-0/minicluster-data/ts-1/wal
uuid: "0b44f08250a042d1a6105506d57e4c6a"
format_stamp: "Formatted at 2026-08-06 03:48:33 on dist-test-slave-6ntk"
server_key: "7aaba10cc4faaf09c27942a205bc7d50"
server_key_iv: "ada5bad631e5520463d02762c94d7bd6"
server_key_version: "kuduclusterkey@0"
I20260806 03:48:33.163415 28668 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988004637995-12279-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988004637995-12279-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988004637995-12279-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:48:33.184634 28668 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:48:33.186976 28668 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:48:33.187372 28668 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:48:33.187791 28668 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 03:48:33.188335 28668 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 03:48:33.188386 28668 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 03:48:33.188410 28668 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 03:48:33.188455 28668 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 03:48:33.199033 28668 rpc_server.cc:307] RPC server started. Bound to: 127.11.253.194:44413
I20260806 03:48:33.199527 28668 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988004637995-12279-0/minicluster-data/ts-1/data/info.pb
I20260806 03:48:33.206970 28853 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.253.194:44413 every 8 connection(s)
I20260806 03:48:33.209499 12279 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskToVWRe/build/release/bin/kudu as pid 28668
I20260806 03:48:33.209630 12279 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988004637995-12279-0/minicluster-data/ts-1/wal/instance
I20260806 03:48:33.209896 12279 external_mini_cluster.cc:1428] Setting key 50818b26eed08523e85368882f96577a
Aug 06 03:48:33 dist-test-slave-6ntk krb5kdc[27666](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988113, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.253.194@KRBTEST.COM for kudu/127.11.253.254@KRBTEST.COM
I20260806 03:48:33.222182 28662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:48:33.207881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.253.194:57747 (local address 127.11.253.254:39491)
0806 03:48:33.207962 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0806 03:48:33.207964 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:48:33.208393 (+ 429us) server_negotiation.cc:407] Connection header received
0806 03:48:33.209211 (+ 818us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:48:33.209213 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:48:33.209231 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:48:33.209264 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:48:33.211041 (+ 1777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:48:33.211810 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:48:33.213325 (+ 1515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:48:33.213549 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:48:33.213647 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:48:33.217892 (+ 4245us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:48:33.217894 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:48:33.217894 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:48:33.217904 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:48:33.220128 (+ 2224us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:48:33.220787 (+ 659us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:48:33.220788 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:48:33.220789 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:48:33.220865 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:48:33.221210 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:48:33.221211 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:48:33.221212 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:48:33.221381 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:48:33.221436 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0806 03:48:33.221922 (+ 486us) server_negotiation.cc:299] Negotiation successful
0806 03:48:33.222041 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260806 03:48:33.222391 28856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:48:33.207882 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:39491 (local address 127.11.253.194:57747)
0806 03:48:33.208209 (+ 327us) negotiation.cc:107] Waiting for socket to connect
0806 03:48:33.208266 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0806 03:48:33.209012 (+ 746us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:48:33.209501 (+ 489us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:48:33.209505 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:48:33.209928 (+ 423us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:48:33.210922 (+ 994us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:48:33.210931 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:48:33.211937 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:48:33.211938 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:48:33.213208 (+ 1270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:48:33.213211 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:48:33.213661 (+ 450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:48:33.213662 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:48:33.213792 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:48:33.214491 (+ 699us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:48:33.214500 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:48:33.217700 (+ 3200us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:48:33.220242 (+ 2542us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:48:33.220252 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:48:33.220255 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:48:33.220653 (+ 398us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:48:33.220952 (+ 299us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:48:33.220953 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:48:33.220953 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:48:33.221118 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:48:33.221495 (+ 377us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:48:33.221503 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:48:33.221694 (+ 191us) client_negotiation.cc:769] Sending connection context
0806 03:48:33.221845 (+ 151us) client_negotiation.cc:240] Negotiation successful
0806 03:48:33.222117 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":253,"thread_start_us":159,"threads_started":1}
I20260806 03:48:33.223243 28854 heartbeater.cc:347] Connected to a master server at 127.11.253.254:39491
WARNING: no policy specified for kudu/127.11.253.195@KRBTEST.COM; defaulting to no policy
I20260806 03:48:33.223373 28854 heartbeater.cc:464] Registering TS with master...
I20260806 03:48:33.223625 28854 heartbeater.cc:511] Master 127.11.253.254:39491 requested a full tablet report, sending...
I20260806 03:48:33.224063 28357 ts_manager.cc:194] Registered new tserver with Master: 0b44f08250a042d1a6105506d57e4c6a (127.11.253.194:44413)
I20260806 03:48:33.224593 28357 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.253.194@KRBTEST.COM'} at 127.11.253.194:57747
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.253.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.253.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.195.keytab.
Entry for principal kudu/127.11.253.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.195.keytab.
I20260806 03:48:33.251277 12279 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskToVWRe/build/release/bin/kudu
/tmp/dist-test-taskToVWRe/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988004637995-12279-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988004637995-12279-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988004637995-12279-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988004637995-12279-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.11.253.212:55111/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.253.195:0
--local_ip_for_outbound_sockets=127.11.253.195
--webserver_interface=127.11.253.195
--webserver_port=0
--tserver_master_addrs=127.11.253.254:39491
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.195.keytab
--principal=kudu/127.11.253.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.253.212:46027
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988004637995-12279-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988004637995-12279-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988004637995-12279-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:48:33.396955 28861 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:48:33.397287 28861 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:48:33.397329 28861 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:48:33.399121 28861 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:48:33.399200 28861 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:48:33.399250 28861 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.253.195
I20260806 03:48:33.401660 28861 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.253.212:46027
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988004637995-12279-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988004637995-12279-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.11.253.212:55111/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.195.keytab
--principal=kudu/127.11.253.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.253.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988004637995-12279-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.253.195
--webserver_port=0
--tserver_master_addrs=127.11.253.254:39491
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28861
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.253.195
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988004637995-12279-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:48:33.401935 28861 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:48:33.402235 28861 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:48:33.409694 28867 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:48:33.409754 28868 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:48:33.410339 28871 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:48:33.410651 28861 server_base.cc:1034] running on GCE node
I20260806 03:48:33.410867 28861 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:48:33.411135 28861 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:48:33.412312 28861 hybrid_clock.cc:648] HybridClock initialized: now 1785988113412286 us; error 49 us; skew 500 ppm
Aug 06 03:48:33 dist-test-slave-6ntk krb5kdc[27666](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988113, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.253.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:48:33.415835 28861 init.cc:377] Logged in from keytab as kudu/127.11.253.195@KRBTEST.COM (short username kudu)
I20260806 03:48:33.416345 28861 webserver.cc:460] Webserver started at http://127.11.253.195:43075/ using document root <none> and password file <none>
I20260806 03:48:33.416548 28861 fs_manager.cc:359] Metadata directory not provided
I20260806 03:48:33.416591 28861 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:48:33.416742 28861 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 03:48:33 dist-test-slave-6ntk krb5kdc[27666](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988113, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.253.195@KRBTEST.COM for HTTP/127.11.253.212@KRBTEST.COM
I20260806 03:48:33.433944 28861 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988004637995-12279-0/minicluster-data/ts-2/data/instance:
uuid: "6c5aa59bc3234941806791e19814be28"
format_stamp: "Formatted at 2026-08-06 03:48:33 on dist-test-slave-6ntk"
server_key: "575a030f5c6b250e5802b848b314b08b"
server_key_iv: "f30c00694a24bc759473d63425a0c8ad"
server_key_version: "kuduclusterkey@0"
I20260806 03:48:33.434522 28861 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988004637995-12279-0/minicluster-data/ts-2/wal/instance:
uuid: "6c5aa59bc3234941806791e19814be28"
format_stamp: "Formatted at 2026-08-06 03:48:33 on dist-test-slave-6ntk"
server_key: "575a030f5c6b250e5802b848b314b08b"
server_key_iv: "f30c00694a24bc759473d63425a0c8ad"
server_key_version: "kuduclusterkey@0"
I20260806 03:48:33.436966 28861 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.002s
I20260806 03:48:33.452678 28879 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:48:33.452976 28861 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20260806 03:48:33.453104 28861 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988004637995-12279-0/minicluster-data/ts-2/data,/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988004637995-12279-0/minicluster-data/ts-2/wal
uuid: "6c5aa59bc3234941806791e19814be28"
format_stamp: "Formatted at 2026-08-06 03:48:33 on dist-test-slave-6ntk"
server_key: "575a030f5c6b250e5802b848b314b08b"
server_key_iv: "f30c00694a24bc759473d63425a0c8ad"
server_key_version: "kuduclusterkey@0"
I20260806 03:48:33.453238 28861 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988004637995-12279-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988004637995-12279-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988004637995-12279-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:48:33.487730 28861 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:48:33.490409 28861 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:48:33.490569 28861 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:48:33.490839 28861 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 03:48:33.491283 28861 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 03:48:33.491335 28861 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 03:48:33.491367 28861 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 03:48:33.491389 28861 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 03:48:33.506744 28861 rpc_server.cc:307] RPC server started. Bound to: 127.11.253.195:46819
I20260806 03:48:33.507284 28861 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988004637995-12279-0/minicluster-data/ts-2/data/info.pb
I20260806 03:48:33.510429 12279 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskToVWRe/build/release/bin/kudu as pid 28861
I20260806 03:48:33.510546 12279 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988004637995-12279-0/minicluster-data/ts-2/wal/instance
I20260806 03:48:33.510758 12279 external_mini_cluster.cc:1428] Setting key 7d70292576410f2472289262993e9aa1
I20260806 03:48:33.518884 29039 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.253.195:46819 every 8 connection(s)
Aug 06 03:48:33 dist-test-slave-6ntk krb5kdc[27666](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988113, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.253.195@KRBTEST.COM for kudu/127.11.253.254@KRBTEST.COM
I20260806 03:48:33.535322 28662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:48:33.519985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.253.195:59371 (local address 127.11.253.254:39491)
0806 03:48:33.520037 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0806 03:48:33.520039 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:48:33.521836 (+ 1797us) server_negotiation.cc:407] Connection header received
0806 03:48:33.522976 (+ 1140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:48:33.522977 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:48:33.522990 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:48:33.523021 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:48:33.524089 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:48:33.524909 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:48:33.526696 (+ 1787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:48:33.527004 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:48:33.527055 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:48:33.531443 (+ 4388us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:48:33.531445 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:48:33.531445 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:48:33.531453 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:48:33.533351 (+ 1898us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:48:33.534113 (+ 762us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:48:33.534115 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:48:33.534116 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:48:33.534180 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:48:33.534530 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:48:33.534531 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:48:33.534531 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:48:33.534673 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:48:33.534732 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0806 03:48:33.535118 (+ 386us) server_negotiation.cc:299] Negotiation successful
0806 03:48:33.535221 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260806 03:48:33.536190 29043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:48:33.519620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:39491 (local address 127.11.253.195:59371)
0806 03:48:33.521606 (+ 1986us) negotiation.cc:107] Waiting for socket to connect
0806 03:48:33.521664 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0806 03:48:33.522844 (+ 1180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:48:33.523242 (+ 398us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:48:33.523247 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:48:33.523538 (+ 291us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:48:33.523969 (+ 431us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:48:33.523976 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:48:33.525033 (+ 1057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:48:33.525033 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:48:33.526584 (+ 1551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:48:33.526588 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:48:33.527155 (+ 567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:48:33.527156 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:48:33.527387 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:48:33.529259 (+ 1872us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:48:33.529269 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:48:33.531322 (+ 2053us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:48:33.533552 (+ 2230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:48:33.533559 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:48:33.533562 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:48:33.534012 (+ 450us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:48:33.534300 (+ 288us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:48:33.534301 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:48:33.534302 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:48:33.534446 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:48:33.534770 (+ 324us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:48:33.534777 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:48:33.534952 (+ 175us) client_negotiation.cc:769] Sending connection context
0806 03:48:33.535680 (+ 728us) client_negotiation.cc:240] Negotiation successful
0806 03:48:33.535859 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1909,"thread_start_us":131,"threads_started":1}
I20260806 03:48:33.536959 29040 heartbeater.cc:347] Connected to a master server at 127.11.253.254:39491
I20260806 03:48:33.537099 29040 heartbeater.cc:464] Registering TS with master...
I20260806 03:48:33.537308 29040 heartbeater.cc:511] Master 127.11.253.254:39491 requested a full tablet report, sending...
I20260806 03:48:33.537762 28357 ts_manager.cc:194] Registered new tserver with Master: 6c5aa59bc3234941806791e19814be28 (127.11.253.195:46819)
I20260806 03:48:33.538295 28357 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.253.195@KRBTEST.COM'} at 127.11.253.195:59371
I20260806 03:48:33.545104 12279 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 03:48:33.559297 28662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:48:33.549373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47444 (local address 127.11.253.254:39491)
0806 03:48:33.549423 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0806 03:48:33.549425 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:48:33.549750 (+ 325us) server_negotiation.cc:407] Connection header received
0806 03:48:33.549863 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:48:33.549864 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:48:33.549896 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:48:33.549931 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:48:33.551125 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:48:33.551876 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:48:33.552974 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:48:33.553269 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:48:33.553313 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:48:33.555531 (+ 2218us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:48:33.555533 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:48:33.555533 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:48:33.555542 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:48:33.557547 (+ 2005us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:48:33.558062 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:48:33.558064 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:48:33.558064 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:48:33.558136 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:48:33.558415 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:48:33.558416 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:48:33.558416 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:48:33.558575 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:48:33.558686 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0806 03:48:33.559049 (+ 363us) server_negotiation.cc:299] Negotiation successful
0806 03:48:33.559145 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260806 03:48:33.562053 28357 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47444:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 03:48:33.565208 28357 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 03:48:33.580916 29061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:48:33.570902 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.195:46819 (local address 127.0.0.1:39206)
0806 03:48:33.572172 (+ 1270us) negotiation.cc:107] Waiting for socket to connect
0806 03:48:33.572195 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0806 03:48:33.572257 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:48:33.572561 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:48:33.572565 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:48:33.572568 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 03:48:33.572649 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:48:33.572651 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:48:33.574636 (+ 1985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:48:33.574638 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:48:33.578422 (+ 3784us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:48:33.578425 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:48:33.580478 (+ 2053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:48:33.580479 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:48:33.580574 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:48:33.580591 (+ 17us) client_negotiation.cc:769] Sending connection context
0806 03:48:33.580702 (+ 111us) client_negotiation.cc:240] Negotiation successful
0806 03:48:33.580792 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1252,"thread_start_us":169,"threads_started":1}
I20260806 03:48:33.588615 29062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:48:33.571124 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39206 (local address 127.11.253.195:46819)
0806 03:48:33.572112 (+ 988us) server_negotiation.cc:206] Beginning negotiation
0806 03:48:33.572118 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 03:48:33.572323 (+ 205us) server_negotiation.cc:407] Connection header received
0806 03:48:33.572375 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:48:33.572378 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:48:33.572396 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:48:33.572750 (+ 354us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 03:48:33.572834 (+ 84us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:48:33.574510 (+ 1676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:48:33.578958 (+ 4448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:48:33.580099 (+ 1141us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:48:33.580288 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:48:33.580393 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0806 03:48:33.588415 (+ 8022us) server_negotiation.cc:299] Negotiation successful
0806 03:48:33.588492 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":945,"thread_start_us":89,"threads_started":1}
I20260806 03:48:33.589356 29059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:48:33.570678 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.194:44413 (local address 127.0.0.1:46884)
0806 03:48:33.571734 (+ 1056us) negotiation.cc:107] Waiting for socket to connect
0806 03:48:33.571781 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0806 03:48:33.571870 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:48:33.576470 (+ 4600us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:48:33.576471 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:48:33.576473 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 03:48:33.576536 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:48:33.576537 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:48:33.582683 (+ 6146us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:48:33.582684 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:48:33.583821 (+ 1137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:48:33.583825 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:48:33.589077 (+ 5252us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:48:33.589078 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:48:33.589161 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:48:33.589167 (+ 6us) client_negotiation.cc:769] Sending connection context
0806 03:48:33.589217 (+ 50us) client_negotiation.cc:240] Negotiation successful
0806 03:48:33.589262 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1020,"thread_start_us":72,"threads_started":1}
I20260806 03:48:33.589632 28930 tablet_service.cc:1467] Processing CreateTablet for tablet f683513de7bf4e45a912c6981341bed8 (DEFAULT_TABLE table=test-table [id=719d514a2bd24730a1b7fadbd85c0070]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 03:48:33.590065 28930 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f683513de7bf4e45a912c6981341bed8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:48:33.591842 29063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:48:33.571206 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.193:41291 (local address 127.0.0.1:35696)
0806 03:48:33.576324 (+ 5118us) negotiation.cc:107] Waiting for socket to connect
0806 03:48:33.576334 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 03:48:33.576381 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:48:33.581857 (+ 5476us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:48:33.581858 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:48:33.581861 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 03:48:33.582533 (+ 672us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:48:33.582537 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:48:33.585561 (+ 3024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:48:33.585562 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:48:33.587296 (+ 1734us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:48:33.587305 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:48:33.591140 (+ 3835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:48:33.591141 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:48:33.591234 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:48:33.591240 (+ 6us) client_negotiation.cc:769] Sending connection context
0806 03:48:33.591292 (+ 52us) client_negotiation.cc:240] Negotiation successful
0806 03:48:33.591332 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5100,"thread_start_us":135,"threads_started":1}
I20260806 03:48:33.592366 29060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:48:33.570819 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46884 (local address 127.11.253.194:44413)
0806 03:48:33.576090 (+ 5271us) server_negotiation.cc:206] Beginning negotiation
0806 03:48:33.576095 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 03:48:33.576107 (+ 12us) server_negotiation.cc:407] Connection header received
0806 03:48:33.576149 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:48:33.576151 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:48:33.576168 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:48:33.576626 (+ 458us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 03:48:33.576715 (+ 89us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:48:33.578006 (+ 1291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:48:33.583992 (+ 5986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:48:33.588263 (+ 4271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:48:33.589780 (+ 1517us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:48:33.589967 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0806 03:48:33.590001 (+ 34us) server_negotiation.cc:299] Negotiation successful
0806 03:48:33.590065 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5225,"thread_start_us":85,"threads_started":1}
I20260806 03:48:33.592871 29064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:48:33.571340 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35696 (local address 127.11.253.193:41291)
0806 03:48:33.581558 (+ 10218us) server_negotiation.cc:206] Beginning negotiation
0806 03:48:33.581563 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 03:48:33.581578 (+ 15us) server_negotiation.cc:407] Connection header received
0806 03:48:33.581631 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:48:33.581633 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:48:33.581653 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:48:33.581749 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 03:48:33.584341 (+ 2592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:48:33.585440 (+ 1099us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:48:33.590230 (+ 4790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:48:33.590959 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:48:33.591045 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:48:33.591098 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0806 03:48:33.592734 (+ 1636us) server_negotiation.cc:299] Negotiation successful
0806 03:48:33.592791 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10173,"thread_start_us":139,"threads_started":1}
I20260806 03:48:33.593251 28745 tablet_service.cc:1467] Processing CreateTablet for tablet f683513de7bf4e45a912c6981341bed8 (DEFAULT_TABLE table=test-table [id=719d514a2bd24730a1b7fadbd85c0070]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 03:48:33.593588 28745 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f683513de7bf4e45a912c6981341bed8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:48:33.593703 28573 tablet_service.cc:1467] Processing CreateTablet for tablet f683513de7bf4e45a912c6981341bed8 (DEFAULT_TABLE table=test-table [id=719d514a2bd24730a1b7fadbd85c0070]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 03:48:33.594074 28573 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f683513de7bf4e45a912c6981341bed8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:48:33.598479 29067 tablet_bootstrap.cc:492] T f683513de7bf4e45a912c6981341bed8 P 87f8ddf1bae0432d844ce4b79fcb15ff: Bootstrap starting.
I20260806 03:48:33.599673 29067 tablet_bootstrap.cc:654] T f683513de7bf4e45a912c6981341bed8 P 87f8ddf1bae0432d844ce4b79fcb15ff: Neither blocks nor log segments found. Creating new log.
I20260806 03:48:33.600374 29067 log.cc:824] T f683513de7bf4e45a912c6981341bed8 P 87f8ddf1bae0432d844ce4b79fcb15ff: Log is configured to *not* fsync() on all Append() calls
I20260806 03:48:33.601735 29067 tablet_bootstrap.cc:492] T f683513de7bf4e45a912c6981341bed8 P 87f8ddf1bae0432d844ce4b79fcb15ff: No bootstrap required, opened a new log
I20260806 03:48:33.601814 29067 ts_tablet_manager.cc:1397] T f683513de7bf4e45a912c6981341bed8 P 87f8ddf1bae0432d844ce4b79fcb15ff: Time spent bootstrapping tablet: real 0.003s user 0.003s sys 0.000s
I20260806 03:48:33.603367 29068 tablet_bootstrap.cc:492] T f683513de7bf4e45a912c6981341bed8 P 6c5aa59bc3234941806791e19814be28: Bootstrap starting.
I20260806 03:48:33.604835 29068 tablet_bootstrap.cc:654] T f683513de7bf4e45a912c6981341bed8 P 6c5aa59bc3234941806791e19814be28: Neither blocks nor log segments found. Creating new log.
I20260806 03:48:33.605144 29068 log.cc:824] T f683513de7bf4e45a912c6981341bed8 P 6c5aa59bc3234941806791e19814be28: Log is configured to *not* fsync() on all Append() calls
I20260806 03:48:33.606117 29065 tablet_bootstrap.cc:492] T f683513de7bf4e45a912c6981341bed8 P 0b44f08250a042d1a6105506d57e4c6a: Bootstrap starting.
I20260806 03:48:33.605675 29067 raft_consensus.cc:348] T f683513de7bf4e45a912c6981341bed8 P 87f8ddf1bae0432d844ce4b79fcb15ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b44f08250a042d1a6105506d57e4c6a" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 44413 } } peers { permanent_uuid: "87f8ddf1bae0432d844ce4b79fcb15ff" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 41291 } } peers { permanent_uuid: "6c5aa59bc3234941806791e19814be28" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 46819 } }
I20260806 03:48:33.606667 29067 raft_consensus.cc:374] T f683513de7bf4e45a912c6981341bed8 P 87f8ddf1bae0432d844ce4b79fcb15ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:48:33.606774 29067 raft_consensus.cc:729] T f683513de7bf4e45a912c6981341bed8 P 87f8ddf1bae0432d844ce4b79fcb15ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 87f8ddf1bae0432d844ce4b79fcb15ff, State: Initialized, Role: FOLLOWER
I20260806 03:48:33.606985 29067 consensus_queue.cc:260] T f683513de7bf4e45a912c6981341bed8 P 87f8ddf1bae0432d844ce4b79fcb15ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b44f08250a042d1a6105506d57e4c6a" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 44413 } } peers { permanent_uuid: "87f8ddf1bae0432d844ce4b79fcb15ff" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 41291 } } peers { permanent_uuid: "6c5aa59bc3234941806791e19814be28" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 46819 } }
I20260806 03:48:33.607312 29065 tablet_bootstrap.cc:654] T f683513de7bf4e45a912c6981341bed8 P 0b44f08250a042d1a6105506d57e4c6a: Neither blocks nor log segments found. Creating new log.
I20260806 03:48:33.607518 29067 ts_tablet_manager.cc:1428] T f683513de7bf4e45a912c6981341bed8 P 87f8ddf1bae0432d844ce4b79fcb15ff: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260806 03:48:33.607695 28660 heartbeater.cc:503] Master 127.11.253.254:39491 was elected leader, sending a full tablet report...
I20260806 03:48:33.607676 29065 log.cc:824] T f683513de7bf4e45a912c6981341bed8 P 0b44f08250a042d1a6105506d57e4c6a: Log is configured to *not* fsync() on all Append() calls
I20260806 03:48:33.610296 29068 tablet_bootstrap.cc:492] T f683513de7bf4e45a912c6981341bed8 P 6c5aa59bc3234941806791e19814be28: No bootstrap required, opened a new log
I20260806 03:48:33.610743 29065 tablet_bootstrap.cc:492] T f683513de7bf4e45a912c6981341bed8 P 0b44f08250a042d1a6105506d57e4c6a: No bootstrap required, opened a new log
I20260806 03:48:33.610890 29068 ts_tablet_manager.cc:1397] T f683513de7bf4e45a912c6981341bed8 P 6c5aa59bc3234941806791e19814be28: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.001s
I20260806 03:48:33.613569 29068 raft_consensus.cc:348] T f683513de7bf4e45a912c6981341bed8 P 6c5aa59bc3234941806791e19814be28 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b44f08250a042d1a6105506d57e4c6a" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 44413 } } peers { permanent_uuid: "87f8ddf1bae0432d844ce4b79fcb15ff" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 41291 } } peers { permanent_uuid: "6c5aa59bc3234941806791e19814be28" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 46819 } }
I20260806 03:48:33.614127 29065 ts_tablet_manager.cc:1397] T f683513de7bf4e45a912c6981341bed8 P 0b44f08250a042d1a6105506d57e4c6a: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.000s
I20260806 03:48:33.618005 29065 raft_consensus.cc:348] T f683513de7bf4e45a912c6981341bed8 P 0b44f08250a042d1a6105506d57e4c6a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b44f08250a042d1a6105506d57e4c6a" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 44413 } } peers { permanent_uuid: "87f8ddf1bae0432d844ce4b79fcb15ff" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 41291 } } peers { permanent_uuid: "6c5aa59bc3234941806791e19814be28" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 46819 } }
I20260806 03:48:33.618304 29068 raft_consensus.cc:374] T f683513de7bf4e45a912c6981341bed8 P 6c5aa59bc3234941806791e19814be28 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:48:33.618427 29068 raft_consensus.cc:729] T f683513de7bf4e45a912c6981341bed8 P 6c5aa59bc3234941806791e19814be28 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c5aa59bc3234941806791e19814be28, State: Initialized, Role: FOLLOWER
I20260806 03:48:33.618649 29068 consensus_queue.cc:260] T f683513de7bf4e45a912c6981341bed8 P 6c5aa59bc3234941806791e19814be28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b44f08250a042d1a6105506d57e4c6a" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 44413 } } peers { permanent_uuid: "87f8ddf1bae0432d844ce4b79fcb15ff" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 41291 } } peers { permanent_uuid: "6c5aa59bc3234941806791e19814be28" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 46819 } }
I20260806 03:48:33.619097 29068 ts_tablet_manager.cc:1428] T f683513de7bf4e45a912c6981341bed8 P 6c5aa59bc3234941806791e19814be28: Time spent starting tablet: real 0.008s user 0.000s sys 0.003s
I20260806 03:48:33.619293 29065 raft_consensus.cc:374] T f683513de7bf4e45a912c6981341bed8 P 0b44f08250a042d1a6105506d57e4c6a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:48:33.619318 29040 heartbeater.cc:503] Master 127.11.253.254:39491 was elected leader, sending a full tablet report...
I20260806 03:48:33.619410 29065 raft_consensus.cc:729] T f683513de7bf4e45a912c6981341bed8 P 0b44f08250a042d1a6105506d57e4c6a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b44f08250a042d1a6105506d57e4c6a, State: Initialized, Role: FOLLOWER
I20260806 03:48:33.619585 29065 consensus_queue.cc:260] T f683513de7bf4e45a912c6981341bed8 P 0b44f08250a042d1a6105506d57e4c6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b44f08250a042d1a6105506d57e4c6a" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 44413 } } peers { permanent_uuid: "87f8ddf1bae0432d844ce4b79fcb15ff" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 41291 } } peers { permanent_uuid: "6c5aa59bc3234941806791e19814be28" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 46819 } }
I20260806 03:48:33.619928 29065 ts_tablet_manager.cc:1428] T f683513de7bf4e45a912c6981341bed8 P 0b44f08250a042d1a6105506d57e4c6a: Time spent starting tablet: real 0.006s user 0.004s sys 0.000s
I20260806 03:48:33.620117 28854 heartbeater.cc:503] Master 127.11.253.254:39491 was elected leader, sending a full tablet report...
W20260806 03:48:33.662707 28661 tablet.cc:2369] T f683513de7bf4e45a912c6981341bed8 P 87f8ddf1bae0432d844ce4b79fcb15ff: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 06 03:48:33 dist-test-slave-6ntk krb5kdc[27666](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988113, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.253.254@KRBTEST.COM for HTTP/127.11.253.212@KRBTEST.COM
W20260806 03:48:33.710017 28855 tablet.cc:2369] T f683513de7bf4e45a912c6981341bed8 P 0b44f08250a042d1a6105506d57e4c6a: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 03:48:33.770476 29042 tablet.cc:2369] T f683513de7bf4e45a912c6981341bed8 P 6c5aa59bc3234941806791e19814be28: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 03:48:33.921998 29071 raft_consensus.cc:482] T f683513de7bf4e45a912c6981341bed8 P 87f8ddf1bae0432d844ce4b79fcb15ff [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 03:48:33.922142 29071 raft_consensus.cc:504] T f683513de7bf4e45a912c6981341bed8 P 87f8ddf1bae0432d844ce4b79fcb15ff [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b44f08250a042d1a6105506d57e4c6a" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 44413 } } peers { permanent_uuid: "87f8ddf1bae0432d844ce4b79fcb15ff" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 41291 } } peers { permanent_uuid: "6c5aa59bc3234941806791e19814be28" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 46819 } }
I20260806 03:48:33.922570 29071 leader_election.cc:288] T f683513de7bf4e45a912c6981341bed8 P 87f8ddf1bae0432d844ce4b79fcb15ff [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0b44f08250a042d1a6105506d57e4c6a (127.11.253.194:44413), 6c5aa59bc3234941806791e19814be28 (127.11.253.195:46819)
I20260806 03:48:33.932148 29060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:48:33.923377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.253.193:55965 (local address 127.11.253.194:44413)
0806 03:48:33.923471 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0806 03:48:33.923473 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:48:33.925944 (+ 2471us) server_negotiation.cc:407] Connection header received
0806 03:48:33.926054 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:48:33.926055 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:48:33.926066 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:48:33.926094 (+ 28us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 03:48:33.926289 (+ 195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:48:33.926975 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:48:33.931235 (+ 4260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:48:33.931753 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:48:33.931807 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:48:33.931817 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0806 03:48:33.932015 (+ 198us) server_negotiation.cc:299] Negotiation successful
0806 03:48:33.932057 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260806 03:48:33.932631 29080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:48:33.922878 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.194:44413 (local address 127.11.253.193:55965)
0806 03:48:33.925823 (+ 2945us) negotiation.cc:107] Waiting for socket to connect
0806 03:48:33.925837 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 03:48:33.925978 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:48:33.926159 (+ 181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:48:33.926160 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:48:33.926162 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 03:48:33.926215 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:48:33.926217 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:48:33.927070 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:48:33.927071 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:48:33.928457 (+ 1386us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:48:33.928460 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:48:33.931877 (+ 3417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:48:33.931878 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:48:33.931934 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:48:33.931938 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 03:48:33.932500 (+ 562us) client_negotiation.cc:240] Negotiation successful
0806 03:48:33.932548 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2918,"thread_start_us":95,"threads_started":1}
I20260806 03:48:33.933205 28790 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f683513de7bf4e45a912c6981341bed8" candidate_uuid: "87f8ddf1bae0432d844ce4b79fcb15ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b44f08250a042d1a6105506d57e4c6a" is_pre_election: true
I20260806 03:48:33.933383 28790 raft_consensus.cc:2454] T f683513de7bf4e45a912c6981341bed8 P 0b44f08250a042d1a6105506d57e4c6a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 87f8ddf1bae0432d844ce4b79fcb15ff in term 0.
I20260806 03:48:33.933678 28502 leader_election.cc:302] T f683513de7bf4e45a912c6981341bed8 P 87f8ddf1bae0432d844ce4b79fcb15ff [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0b44f08250a042d1a6105506d57e4c6a, 87f8ddf1bae0432d844ce4b79fcb15ff; no voters:
I20260806 03:48:33.933842 29071 raft_consensus.cc:2781] T f683513de7bf4e45a912c6981341bed8 P 87f8ddf1bae0432d844ce4b79fcb15ff [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 03:48:33.933907 29071 raft_consensus.cc:482] T f683513de7bf4e45a912c6981341bed8 P 87f8ddf1bae0432d844ce4b79fcb15ff [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 03:48:33.933930 29071 raft_consensus.cc:3037] T f683513de7bf4e45a912c6981341bed8 P 87f8ddf1bae0432d844ce4b79fcb15ff [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:48:33.934510 29062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:48:33.923425 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.253.193:48527 (local address 127.11.253.195:46819)
0806 03:48:33.923504 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0806 03:48:33.923506 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:48:33.928642 (+ 5136us) server_negotiation.cc:407] Connection header received
0806 03:48:33.928740 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:48:33.928741 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:48:33.928751 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:48:33.928781 (+ 30us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 03:48:33.928971 (+ 190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:48:33.929643 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:48:33.933094 (+ 3451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:48:33.933635 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:48:33.933705 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:48:33.933713 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0806 03:48:33.934395 (+ 682us) server_negotiation.cc:299] Negotiation successful
0806 03:48:33.934437 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260806 03:48:33.934928 29077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:48:33.923062 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.195:46819 (local address 127.11.253.193:48527)
0806 03:48:33.928583 (+ 5521us) negotiation.cc:107] Waiting for socket to connect
0806 03:48:33.928593 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 03:48:33.928671 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:48:33.928841 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:48:33.928842 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:48:33.928843 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 03:48:33.928908 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:48:33.928909 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:48:33.929896 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:48:33.929896 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:48:33.931107 (+ 1211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:48:33.931110 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:48:33.934192 (+ 3082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:48:33.934193 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:48:33.934275 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:48:33.934279 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 03:48:33.934766 (+ 487us) client_negotiation.cc:240] Negotiation successful
0806 03:48:33.934809 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5503,"thread_start_us":62,"threads_started":1}
I20260806 03:48:33.935050 29071 raft_consensus.cc:504] T f683513de7bf4e45a912c6981341bed8 P 87f8ddf1bae0432d844ce4b79fcb15ff [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b44f08250a042d1a6105506d57e4c6a" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 44413 } } peers { permanent_uuid: "87f8ddf1bae0432d844ce4b79fcb15ff" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 41291 } } peers { permanent_uuid: "6c5aa59bc3234941806791e19814be28" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 46819 } }
I20260806 03:48:33.935287 29071 leader_election.cc:288] T f683513de7bf4e45a912c6981341bed8 P 87f8ddf1bae0432d844ce4b79fcb15ff [CANDIDATE]: Term 1 election: Requested vote from peers 0b44f08250a042d1a6105506d57e4c6a (127.11.253.194:44413), 6c5aa59bc3234941806791e19814be28 (127.11.253.195:46819)
I20260806 03:48:33.935523 28790 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f683513de7bf4e45a912c6981341bed8" candidate_uuid: "87f8ddf1bae0432d844ce4b79fcb15ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b44f08250a042d1a6105506d57e4c6a"
I20260806 03:48:33.935598 28790 raft_consensus.cc:3037] T f683513de7bf4e45a912c6981341bed8 P 0b44f08250a042d1a6105506d57e4c6a [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:48:33.936507 28790 raft_consensus.cc:2454] T f683513de7bf4e45a912c6981341bed8 P 0b44f08250a042d1a6105506d57e4c6a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 87f8ddf1bae0432d844ce4b79fcb15ff in term 1.
I20260806 03:48:33.936738 28502 leader_election.cc:302] T f683513de7bf4e45a912c6981341bed8 P 87f8ddf1bae0432d844ce4b79fcb15ff [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0b44f08250a042d1a6105506d57e4c6a, 87f8ddf1bae0432d844ce4b79fcb15ff; no voters:
I20260806 03:48:33.936857 29071 raft_consensus.cc:2781] T f683513de7bf4e45a912c6981341bed8 P 87f8ddf1bae0432d844ce4b79fcb15ff [term 1 FOLLOWER]: Leader election won for term 1
I20260806 03:48:33.937047 29071 raft_consensus.cc:686] T f683513de7bf4e45a912c6981341bed8 P 87f8ddf1bae0432d844ce4b79fcb15ff [term 1 LEADER]: Becoming Leader. State: Replica: 87f8ddf1bae0432d844ce4b79fcb15ff, State: Running, Role: LEADER
I20260806 03:48:33.937142 29071 consensus_queue.cc:237] T f683513de7bf4e45a912c6981341bed8 P 87f8ddf1bae0432d844ce4b79fcb15ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b44f08250a042d1a6105506d57e4c6a" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 44413 } } peers { permanent_uuid: "87f8ddf1bae0432d844ce4b79fcb15ff" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 41291 } } peers { permanent_uuid: "6c5aa59bc3234941806791e19814be28" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 46819 } }
I20260806 03:48:33.938024 28357 catalog_manager.cc:5495] T f683513de7bf4e45a912c6981341bed8 P 87f8ddf1bae0432d844ce4b79fcb15ff reported cstate change: term changed from 0 to 1, leader changed from <none> to 87f8ddf1bae0432d844ce4b79fcb15ff (127.11.253.193). New cstate: current_term: 1 leader_uuid: "87f8ddf1bae0432d844ce4b79fcb15ff" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b44f08250a042d1a6105506d57e4c6a" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 44413 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "87f8ddf1bae0432d844ce4b79fcb15ff" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 41291 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c5aa59bc3234941806791e19814be28" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 46819 } health_report { overall_health: UNKNOWN } } }
I20260806 03:48:33.938915 28970 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f683513de7bf4e45a912c6981341bed8" candidate_uuid: "87f8ddf1bae0432d844ce4b79fcb15ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c5aa59bc3234941806791e19814be28" is_pre_election: true
I20260806 03:48:33.939101 28970 raft_consensus.cc:2454] T f683513de7bf4e45a912c6981341bed8 P 6c5aa59bc3234941806791e19814be28 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 87f8ddf1bae0432d844ce4b79fcb15ff in term 0.
I20260806 03:48:33.939275 28968 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f683513de7bf4e45a912c6981341bed8" candidate_uuid: "87f8ddf1bae0432d844ce4b79fcb15ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c5aa59bc3234941806791e19814be28"
I20260806 03:48:33.939373 28968 raft_consensus.cc:3037] T f683513de7bf4e45a912c6981341bed8 P 6c5aa59bc3234941806791e19814be28 [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:48:33.940388 28968 raft_consensus.cc:2454] T f683513de7bf4e45a912c6981341bed8 P 6c5aa59bc3234941806791e19814be28 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 87f8ddf1bae0432d844ce4b79fcb15ff in term 1.
Aug 06 03:48:34 dist-test-slave-6ntk krb5kdc[27666](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988113, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.253.254@KRBTEST.COM for HTTP/127.11.253.212@KRBTEST.COM
I20260806 03:48:34.024570 29064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:48:34.017365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35706 (local address 127.11.253.193:41291)
0806 03:48:34.017410 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0806 03:48:34.017411 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 03:48:34.017421 (+ 10us) server_negotiation.cc:407] Connection header received
0806 03:48:34.017469 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:48:34.017470 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:48:34.017494 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:48:34.017577 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 03:48:34.021519 (+ 3942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:48:34.022285 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:48:34.023351 (+ 1066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:48:34.023633 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:48:34.023707 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:48:34.023950 (+ 243us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 03:48:34.024103 (+ 153us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 03:48:34.024180 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0806 03:48:34.024451 (+ 271us) server_negotiation.cc:299] Negotiation successful
0806 03:48:34.024474 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 03:48:34.027688 28968 raft_consensus.cc:1261] T f683513de7bf4e45a912c6981341bed8 P 6c5aa59bc3234941806791e19814be28 [term 1 FOLLOWER]: Refusing update from remote peer 87f8ddf1bae0432d844ce4b79fcb15ff: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 03:48:34.028177 29081 consensus_queue.cc:1035] T f683513de7bf4e45a912c6981341bed8 P 87f8ddf1bae0432d844ce4b79fcb15ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c5aa59bc3234941806791e19814be28" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 46819 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 03:48:34.029098 28790 raft_consensus.cc:1261] T f683513de7bf4e45a912c6981341bed8 P 0b44f08250a042d1a6105506d57e4c6a [term 1 FOLLOWER]: Refusing update from remote peer 87f8ddf1bae0432d844ce4b79fcb15ff: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 03:48:34.029621 29081 consensus_queue.cc:1035] T f683513de7bf4e45a912c6981341bed8 P 87f8ddf1bae0432d844ce4b79fcb15ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "0b44f08250a042d1a6105506d57e4c6a" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 44413 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 03:48:34.033861 29086 mvcc.cc:205] Tried to move back new op lower bound from 7315407315053821952 to 7315407314687119360. Current Snapshot: MvccSnapshot[applied={T|T < 7315407315053821952 or (T in {7315407315053821952})}]
I20260806 03:48:34.034034 29088 mvcc.cc:205] Tried to move back new op lower bound from 7315407315053821952 to 7315407314687119360. Current Snapshot: MvccSnapshot[applied={T|T < 7315407315053821952 or (T in {7315407315053821952})}]
I20260806 03:48:34.035991 28350 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47444:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 03:48:34.036099 28350 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47444:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 03:48:34.037155 28350 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8f31c12182a5454db3f2321acd89cf00: Sending DeleteTablet for 3 replicas of tablet f683513de7bf4e45a912c6981341bed8
I20260806 03:48:34.037590 28930 tablet_service.cc:1514] Processing DeleteTablet for tablet f683513de7bf4e45a912c6981341bed8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 03:48:34 UTC) from {username='kudu', principal='kudu/127.11.253.254@KRBTEST.COM'} at 127.0.0.1:39206
I20260806 03:48:34.037858 28745 tablet_service.cc:1514] Processing DeleteTablet for tablet f683513de7bf4e45a912c6981341bed8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 03:48:34 UTC) from {username='kudu', principal='kudu/127.11.253.254@KRBTEST.COM'} at 127.0.0.1:46884
I20260806 03:48:34.038331 28573 tablet_service.cc:1514] Processing DeleteTablet for tablet f683513de7bf4e45a912c6981341bed8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 03:48:34 UTC) from {username='kudu', principal='kudu/127.11.253.254@KRBTEST.COM'} at 127.0.0.1:35696
I20260806 03:48:34.039093 29101 tablet_replica.cc:330] T f683513de7bf4e45a912c6981341bed8 P 0b44f08250a042d1a6105506d57e4c6a: stopping tablet replica
I20260806 03:48:34.039211 29101 raft_consensus.cc:2229] T f683513de7bf4e45a912c6981341bed8 P 0b44f08250a042d1a6105506d57e4c6a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 03:48:34.039340 29101 raft_consensus.cc:2258] T f683513de7bf4e45a912c6981341bed8 P 0b44f08250a042d1a6105506d57e4c6a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 03:48:34.039490 29100 tablet_replica.cc:330] T f683513de7bf4e45a912c6981341bed8 P 6c5aa59bc3234941806791e19814be28: stopping tablet replica
I20260806 03:48:34.039589 29100 raft_consensus.cc:2229] T f683513de7bf4e45a912c6981341bed8 P 6c5aa59bc3234941806791e19814be28 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 03:48:34.039690 29100 raft_consensus.cc:2258] T f683513de7bf4e45a912c6981341bed8 P 6c5aa59bc3234941806791e19814be28 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 03:48:34.039840 29102 tablet_replica.cc:330] T f683513de7bf4e45a912c6981341bed8 P 87f8ddf1bae0432d844ce4b79fcb15ff: stopping tablet replica
I20260806 03:48:34.039922 29102 raft_consensus.cc:2229] T f683513de7bf4e45a912c6981341bed8 P 87f8ddf1bae0432d844ce4b79fcb15ff [term 1 LEADER]: Raft consensus shutting down.
I20260806 03:48:34.040011 29102 raft_consensus.cc:2258] T f683513de7bf4e45a912c6981341bed8 P 87f8ddf1bae0432d844ce4b79fcb15ff [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 03:48:34.042064 29100 ts_tablet_manager.cc:1905] T f683513de7bf4e45a912c6981341bed8 P 6c5aa59bc3234941806791e19814be28: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 03:48:34.043902 29100 ts_tablet_manager.cc:1918] T f683513de7bf4e45a912c6981341bed8 P 6c5aa59bc3234941806791e19814be28: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 03:48:34.043984 29100 log.cc:1196] T f683513de7bf4e45a912c6981341bed8 P 6c5aa59bc3234941806791e19814be28: Deleting WAL directory at /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988004637995-12279-0/minicluster-data/ts-2/wal/wals/f683513de7bf4e45a912c6981341bed8
I20260806 03:48:34.044260 29100 ts_tablet_manager.cc:1939] T f683513de7bf4e45a912c6981341bed8 P 6c5aa59bc3234941806791e19814be28: Deleting consensus metadata
I20260806 03:48:34.046087 28335 catalog_manager.cc:4841] TS 6c5aa59bc3234941806791e19814be28 (127.11.253.195:46819): tablet f683513de7bf4e45a912c6981341bed8 (table test-table [id=719d514a2bd24730a1b7fadbd85c0070]) successfully deleted
I20260806 03:48:34.046465 12279 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskToVWRe/build/release/bin/kudu with pid 28474
I20260806 03:48:34.054981 29101 ts_tablet_manager.cc:1905] T f683513de7bf4e45a912c6981341bed8 P 0b44f08250a042d1a6105506d57e4c6a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 03:48:34.057291 29101 ts_tablet_manager.cc:1918] T f683513de7bf4e45a912c6981341bed8 P 0b44f08250a042d1a6105506d57e4c6a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 03:48:34.057799 29101 log.cc:1196] T f683513de7bf4e45a912c6981341bed8 P 0b44f08250a042d1a6105506d57e4c6a: Deleting WAL directory at /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988004637995-12279-0/minicluster-data/ts-1/wal/wals/f683513de7bf4e45a912c6981341bed8
I20260806 03:48:34.058220 29101 ts_tablet_manager.cc:1939] T f683513de7bf4e45a912c6981341bed8 P 0b44f08250a042d1a6105506d57e4c6a: Deleting consensus metadata
I20260806 03:48:34.058840 28335 catalog_manager.cc:4841] TS 0b44f08250a042d1a6105506d57e4c6a (127.11.253.194:44413): tablet f683513de7bf4e45a912c6981341bed8 (table test-table [id=719d514a2bd24730a1b7fadbd85c0070]) successfully deleted
W20260806 03:48:34.061998 28337 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.253.193:41291 (error 108)
I20260806 03:48:34.062552 29063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:48:34.062406 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.193:41291 (local address 127.0.0.1:35720)
0806 03:48:34.062467 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0806 03:48:34.062508 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.253.193:41291: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260806 03:48:34.062665 28337 catalog_manager.cc:4584] TS 87f8ddf1bae0432d844ce4b79fcb15ff (127.11.253.193:41291): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet f683513de7bf4e45a912c6981341bed8: Network error: Client connection negotiation failed: client connection to 127.11.253.193:41291: connect: Connection refused (error 111)
I20260806 03:48:34.063038 12279 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskToVWRe/build/release/bin/kudu with pid 28668
I20260806 03:48:34.075738 12279 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskToVWRe/build/release/bin/kudu with pid 28861
I20260806 03:48:34.086814 12279 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskToVWRe/build/release/bin/kudu with pid 28311
I20260806 03:48:34.110399 12279 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260806 03:48:34.466089 12279 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260806 03:48:34.466198 12279 mini_ranger.cc:66] Stopping Ranger...
I20260806 03:48:34.821760 12279 mini_ranger.cc:68] Stopped Ranger
2026-08-06 03:48:34.822 UTC [27710] LOG: received smart shutdown request
2026-08-06 03:48:34.826 UTC [27710] LOG: background worker "logical replication launcher" (PID 27725) exited with exit code 1
2026-08-06 03:48:34.826 UTC [27718] LOG: shutting down
2026-08-06 03:48:35.026 UTC [27710] LOG: database system is shut down
2026-08-06T03:48:35Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (51026 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988004637995-12279-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 03:48:35 dist-test-slave-6ntk krb5kdc[29119](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 03:48:35 dist-test-slave-6ntk krb5kdc[29119](info): set up 2 sockets
Aug 06 03:48:35 dist-test-slave-6ntk krb5kdc[29119](info): commencing operation
krb5kdc: starting...
W20260806 03:48:36.397228 12279 mini_kdc.cc:121] Time spent starting KDC: real 1.226s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988004637995-12279-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988004637995-12279-0/krb5kdc/test-user.keytab.
Aug 06 03:48:36 dist-test-slave-6ntk krb5kdc[29119](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988116, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T03:48:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T03:48:36Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.11.253.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.11.253.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.11.253.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988004637995-12279-0/krb5kdc/rangeradmin_127.11.253.212@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.11.253.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988004637995-12279-0/krb5kdc/rangeradmin_127.11.253.212@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.11.253.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.11.253.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.11.253.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988004637995-12279-0/krb5kdc/rangerlookup_127.11.253.212@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.11.253.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988004637995-12279-0/krb5kdc/rangerlookup_127.11.253.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.11.253.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.11.253.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.11.253.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988004637995-12279-0/krb5kdc/HTTP_127.11.253.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.11.253.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988004637995-12279-0/krb5kdc/HTTP_127.11.253.212@KRBTEST.COM.keytab.
I20260806 03:48:36.585253 12279 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988004637995-12279-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskToVWRe/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988004637995-12279-0/minicluster-data/postgres -l logfile start
2026-08-06 03:48:39.239 UTC [29162] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-06 03:48:39.239 UTC [29162] LOG: listening on IPv4 address "127.11.253.212", port 36813
2026-08-06 03:48:39.242 UTC [29162] LOG: listening on Unix socket "/tmp/.s.PGSQL.36813"
2026-08-06 03:48:39.279 UTC [29165] LOG: database system was shut down at 2026-08-06 03:48:37 UTC
2026-08-06 03:48:39.284 UTC [29162] LOG: database system is ready to accept connections
I20260806 03:48:40.391265 12279 mini_postgres.cc:96] Postgres bound to 36813
2026-08-06 03:48:40.398 UTC [29180] FATAL: database "slave" does not exist
127.11.253.212:36813 - accepting connections
I20260806 03:48:40.399222 12279 mini_ranger.cc:161] Starting Ranger...
I20260806 03:48:40.415496 12279 mini_ranger.cc:84] Created miniranger Postgres user
I20260806 03:48:40.914489 12279 mini_ranger.cc:87] Created ranger Postgres database
I20260806 03:48:40.914587 12279 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988004637995-12279-0/minicluster-data/ranger-admin
2026-08-06 03:48:41,234 [I] DB FLAVOR :POSTGRES
2026-08-06 03:48:41,234 [I] --------- Verifying Ranger DB connection ---------
2026-08-06 03:48:41,235 [I] Checking connection..
2026-08-06 03:48:41,235 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskToVWRe/build/release/bin/postgresql.jar:/tmp/dist-test-taskToVWRe/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.253.212:36813/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-06 03:48:41,503 [I] Checking connection passed.
2026-08-06 03:48:41,504 [I] --------- Verifying version history table ---------
2026-08-06 03:48:41,504 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskToVWRe/build/release/bin/postgresql.jar:/tmp/dist-test-taskToVWRe/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.253.212:36813/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 03:48:41,766 [I] Table x_db_version_h does not exist in database ranger
2026-08-06 03:48:41,767 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskToVWRe/build/release/bin/postgresql.jar:/tmp/dist-test-taskToVWRe/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.253.212:36813/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 03:48:42,055 [I] Table x_db_version_h does not exist in database ranger
2026-08-06 03:48:42,056 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-06 03:48:42,056 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskToVWRe/build/release/bin/postgresql.jar:/tmp/dist-test-taskToVWRe/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.253.212:36813/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskToVWRe/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-06 03:48:42.292 UTC [29276] WARNING: there is no transaction in progress
2026-08-06 03:48:42,301 [I] create_dbversion_catalog.sql file imported successfully
2026-08-06 03:48:42,301 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskToVWRe/build/release/bin/postgresql.jar:/tmp/dist-test-taskToVWRe/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.253.212:36813/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 03:48:42,582 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-06 03:48:42,583 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-06 03:48:42,583 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskToVWRe/build/release/bin/postgresql.jar:/tmp/dist-test-taskToVWRe/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.253.212:36813/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-06 03:48:42,828 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskToVWRe/build/release/bin/postgresql.jar:/tmp/dist-test-taskToVWRe/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.253.212:36813/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-06 03:48:43,097 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskToVWRe/build/release/bin/postgresql.jar:/tmp/dist-test-taskToVWRe/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.253.212:36813/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-6ntk.c.gcp-upstream.internal','N') ;"
2026-08-06 03:48:43,365 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskToVWRe/build/release/bin/postgresql.jar:/tmp/dist-test-taskToVWRe/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.253.212:36813/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-06 03:48:43,635 [I] Table x_portal_user does not exist in database ranger
2026-08-06 03:48:43,636 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskToVWRe/build/release/bin/postgresql.jar:/tmp/dist-test-taskToVWRe/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.253.212:36813/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-06 03:48:43,895 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-06 03:48:43,896 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskToVWRe/build/release/bin/postgresql.jar:/tmp/dist-test-taskToVWRe/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.253.212:36813/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-06 03:48:44,191 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskToVWRe/build/release/bin/postgresql.jar:/tmp/dist-test-taskToVWRe/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.253.212:36813/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-06 03:48:44,466 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-06 03:48:44,466 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskToVWRe/build/release/bin/postgresql.jar:/tmp/dist-test-taskToVWRe/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.253.212:36813/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskToVWRe/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-06 03:48:45.526 UTC [29420] WARNING: there is no transaction in progress
2026-08-06 03:48:45.544 UTC [29420] WARNING: there is no transaction in progress
2026-08-06 03:48:45.562 UTC [29420] WARNING: there is no transaction in progress
2026-08-06 03:48:45.579 UTC [29420] WARNING: there is no transaction in progress
2026-08-06 03:48:45.598 UTC [29420] WARNING: there is no transaction in progress
2026-08-06 03:48:45.618 UTC [29420] WARNING: there is no transaction in progress
2026-08-06 03:48:45.679 UTC [29420] WARNING: there is no transaction in progress
2026-08-06 03:48:45.687 UTC [29420] WARNING: there is no transaction in progress
2026-08-06 03:48:45.703 UTC [29420] WARNING: there is no transaction in progress
2026-08-06 03:48:45.714 UTC [29420] WARNING: there is no transaction in progress
2026-08-06 03:48:45.725 UTC [29420] WARNING: there is no transaction in progress
2026-08-06 03:48:45.741 UTC [29420] WARNING: there is no transaction in progress
2026-08-06 03:48:45.752 UTC [29420] WARNING: there is no transaction in progress
2026-08-06 03:48:45.759 UTC [29420] WARNING: there is no transaction in progress
2026-08-06 03:48:45.767 UTC [29420] WARNING: there is no transaction in progress
2026-08-06 03:48:45.775 UTC [29420] WARNING: there is no transaction in progress
2026-08-06 03:48:46,539 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-06 03:48:46,539 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskToVWRe/build/release/bin/postgresql.jar:/tmp/dist-test-taskToVWRe/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.253.212:36813/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-06 03:48:46,775 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-06 03:48:46,775 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskToVWRe/build/release/bin/postgresql.jar:/tmp/dist-test-taskToVWRe/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.253.212:36813/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-06 03:48:47,049 [I] Table x_portal_user already exists in database 'ranger'
2026-08-06 03:48:47,050 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskToVWRe/build/release/bin/postgresql.jar:/tmp/dist-test-taskToVWRe/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.253.212:36813/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-06 03:48:47,309 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-06 03:48:47,309 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskToVWRe/build/release/bin/postgresql.jar:/tmp/dist-test-taskToVWRe/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.253.212:36813/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-06 03:48:47,561 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskToVWRe/build/release/bin/postgresql.jar:/tmp/dist-test-taskToVWRe/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.253.212:36813/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-06 03:48:47,839 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskToVWRe/build/release/bin/postgresql.jar:/tmp/dist-test-taskToVWRe/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.253.212:36813/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-6ntk.c.gcp-upstream.internal';"
2026-08-06 03:48:48,103 [I] CORE_DB_SCHEMA import status has been updated
2026-08-06 03:48:48,104 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskToVWRe/build/release/bin/postgresql.jar:/tmp/dist-test-taskToVWRe/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.253.212:36813/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-06 03:48:48,362 [I] DB_PATCHES have already been applied
I20260806 03:48:48.368209 12279 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988004637995-12279-0/minicluster-data/ranger-admin:/tmp/dist-test-taskToVWRe/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskToVWRe/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskToVWRe/build/release/bin/postgresql.jar:/tmp/dist-test-taskToVWRe/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260806 03:48:48.368276 12279 mini_ranger.cc:193] Using host: 127.11.253.212
I20260806 03:48:48.372557 12279 mini_ranger.cc:227] Ranger admin URL: http://127.11.253.212:49281
Aug 06, 2026 3:48:48 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 06, 2026 3:48:48 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 06, 2026 3:48:48 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988004637995-12279-0/minicluster-data/ranger-admin/ews/webapp
Aug 06, 2026 3:48:48 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988004637995-12279-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 06, 2026 3:48:48 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988004637995-12279-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 06, 2026 3:48:49 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988004637995-12279-0/minicluster-data/ranger-admin/ews/webapp].
Aug 06, 2026 3:48:49 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260806 03:48:49.702440 12279 mini_ranger.cc:160] Time spent starting Ranger: real 9.303s user 0.004s sys 0.007s
Aug 06, 2026 3:49:10 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 06, 2026 3:49:10 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Aug 06, 2026 3:49:10 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 06, 2026 3:49:10 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 06, 2026 3:49:10 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 06, 2026 3:49:10 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 06, 2026 3:49:10 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 06, 2026 3:49:10 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 06, 2026 3:49:10 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 06, 2026 3:49:10 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 06, 2026 3:49:10 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 06, 2026 3:49:10 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 06, 2026 3:49:10 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 06, 2026 3:49:10 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 06, 2026 3:49:10 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 06, 2026 3:49:10 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 06, 2026 3:49:10 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 06, 2026 3:49:11 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260806 03:49:11.683866 12279 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.11.253.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.11.253.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.11.253.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988004637995-12279-0/krb5kdc/rangerkms_127.11.253.212@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.11.253.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988004637995-12279-0/krb5kdc/rangerkms_127.11.253.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.11.253.212@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.11.253.212@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.11.253.212@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988004637995-12279-0/krb5kdc/HTTP_127.11.253.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.11.253.212@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988004637995-12279-0/krb5kdc/HTTP_127.11.253.212@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 06 03:49:11 dist-test-slave-6ntk krb5kdc[29119](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988151, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260806 03:49:11.779597 12279 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260806 03:49:11.791075 12279 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260806 03:49:12.791472 12279 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260806 03:49:12.791576 12279 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988004637995-12279-0/minicluster-data/ranger-kms
I20260806 03:49:12.791596 12279 mini_ranger_kms.cc:227] Using postgres at 127.11.253.212:36813
2026-08-06 03:49:12,975 [I] DB FLAVOR :POSTGRES
2026-08-06 03:49:12,975 [I] --------- Verifying Ranger DB connection ---------
2026-08-06 03:49:12,975 [I] Checking connection
2026-08-06 03:49:12,975 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskToVWRe/build/release/bin/postgresql.jar:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988004637995-12279-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.253.212:36813/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-06 03:49:13,216 [I] connection success
2026-08-06 03:49:13,217 [I] --------- Verifying Ranger DB tables ---------
2026-08-06 03:49:13,217 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-06 03:49:13,217 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskToVWRe/build/release/bin/postgresql.jar:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988004637995-12279-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.253.212:36813/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-06 03:49:13,495 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-06 03:49:13,496 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-06 03:49:13,496 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-06 03:49:13,496 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskToVWRe/build/release/bin/postgresql.jar:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988004637995-12279-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.253.212:36813/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988004637995-12279-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-06 03:49:13,801 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260806 03:49:14.355947 12279 mini_ranger_kms.cc:310] Created kms service
I20260806 03:49:14.531463 12279 mini_ranger_kms.cc:326] Created kudu user
I20260806 03:49:14.579042 12279 mini_ranger_kms.cc:343] Created rangerkms user
Aug 06 03:49:14 dist-test-slave-6ntk krb5kdc[29119](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988151, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.11.253.212@KRBTEST.COM
I20260806 03:49:14.999142 12279 mini_ranger_kms.cc:384] Added ranger policy
I20260806 03:49:14.999257 12279 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988004637995-12279-0/minicluster-data/ranger-kms:/tmp/dist-test-taskToVWRe/build/release/bin/postgresql.jar:/tmp/dist-test-taskToVWRe/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskToVWRe/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260806 03:49:14.999279 12279 mini_ranger_kms.cc:240] Using host: 127.11.253.212
I20260806 03:49:15.003806 12279 mini_ranger_kms.cc:276] Ranger KMS PID: 29667
I20260806 03:49:15.003909 12279 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.11.253.212:34205
Aug 06, 2026 3:49:16 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 06, 2026 3:49:16 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 06, 2026 3:49:16 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988004637995-12279-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 06, 2026 3:49:16 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988004637995-12279-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 06, 2026 3:49:16 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988004637995-12279-0/minicluster-data/ranger-kms/ews/webapp].
Aug 06, 2026 3:49:16 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 06 03:49:17 dist-test-slave-6ntk krb5kdc[29119](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1785988157, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.11.253.212@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:49:17.747252 12279 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.968s user 0.008s sys 0.158s
I20260806 03:49:17.747385 12279 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260806 03:49:17.747437 12279 mini_ranger_kms.cc:401] 127.11.253.212:34205/kms/v1/keys
Aug 06 03:49:18 dist-test-slave-6ntk krb5kdc[29119](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988157, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.11.253.212@KRBTEST.COM for HTTP/127.11.253.212@KRBTEST.COM
Aug 06 03:49:18 dist-test-slave-6ntk krb5kdc[29119](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988157, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.11.253.212@KRBTEST.COM for HTTP/127.11.253.212@KRBTEST.COM
Aug 06 03:49:20 dist-test-slave-6ntk krb5kdc[29119](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988160, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.11.253.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.253.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.253.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.254.keytab.
Entry for principal kudu/127.11.253.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.254.keytab.
I20260806 03:49:20.810735 12279 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskToVWRe/build/release/bin/kudu
/tmp/dist-test-taskToVWRe/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988004637995-12279-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988004637995-12279-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988004637995-12279-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988004637995-12279-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.253.254:35823
--webserver_interface=127.11.253.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.254.keytab
--principal=kudu/127.11.253.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.253.212:38447
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.253.254:35823
--ranger_config_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988004637995-12279-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.11.253.212:34205/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988004637995-12279-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988004637995-12279-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988004637995-12279-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:49:20.895351 29739 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:49:20.895644 29739 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:49:20.895668 29739 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:49:20.896836 29739 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 03:49:20.896879 29739 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:49:20.896893 29739 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:49:20.896906 29739 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 03:49:20.896917 29739 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 03:49:20.898509 29739 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.253.212:38447
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988004637995-12279-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988004637995-12279-0/minicluster-data/master-0/wal
--ranger_kms_url=127.11.253.212:34205/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.11.253.254:35823
--ranger_config_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988004637995-12279-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.254.keytab
--principal=kudu/127.11.253.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.253.254:35823
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988004637995-12279-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.253.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29739
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988004637995-12279-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:49:20.898751 29739 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:49:20.899008 29739 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:49:20.904940 29749 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:49:20.904949 29746 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:49:20.905023 29747 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:49:20.905344 29739 server_base.cc:1034] running on GCE node
I20260806 03:49:20.905629 29739 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:49:20.906015 29739 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:49:20.907187 29739 hybrid_clock.cc:648] HybridClock initialized: now 1785988160907165 us; error 40 us; skew 500 ppm
Aug 06 03:49:20 dist-test-slave-6ntk krb5kdc[29119](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988160, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.253.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:49:20.914141 29739 init.cc:377] Logged in from keytab as kudu/127.11.253.254@KRBTEST.COM (short username kudu)
I20260806 03:49:20.914749 29739 webserver.cc:460] Webserver started at http://127.11.253.254:40951/ using document root <none> and password file <none>
I20260806 03:49:20.915011 29739 fs_manager.cc:359] Metadata directory not provided
I20260806 03:49:20.915066 29739 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:49:20.915207 29739 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 03:49:20 dist-test-slave-6ntk krb5kdc[29119](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988160, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.253.254@KRBTEST.COM for HTTP/127.11.253.212@KRBTEST.COM
I20260806 03:49:20.976290 29739 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988004637995-12279-0/minicluster-data/master-0/data/instance:
uuid: "f648b4f531da41c9a52f2a7b07c56fe3"
format_stamp: "Formatted at 2026-08-06 03:49:20 on dist-test-slave-6ntk"
server_key: "2b68e3cf447d7c5858904eeb9810ba0a"
server_key_iv: "3c1446471ac9ded099ffc3d13af54de2"
server_key_version: "kuduclusterkey@0"
I20260806 03:49:20.976719 29739 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988004637995-12279-0/minicluster-data/master-0/wal/instance:
uuid: "f648b4f531da41c9a52f2a7b07c56fe3"
format_stamp: "Formatted at 2026-08-06 03:49:20 on dist-test-slave-6ntk"
server_key: "2b68e3cf447d7c5858904eeb9810ba0a"
server_key_iv: "3c1446471ac9ded099ffc3d13af54de2"
server_key_version: "kuduclusterkey@0"
I20260806 03:49:20.978457 29739 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.004s sys 0.000s
I20260806 03:49:20.993445 29758 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:49:20.993707 29739 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 03:49:20.993829 29739 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988004637995-12279-0/minicluster-data/master-0/data,/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988004637995-12279-0/minicluster-data/master-0/wal
uuid: "f648b4f531da41c9a52f2a7b07c56fe3"
format_stamp: "Formatted at 2026-08-06 03:49:20 on dist-test-slave-6ntk"
server_key: "2b68e3cf447d7c5858904eeb9810ba0a"
server_key_iv: "3c1446471ac9ded099ffc3d13af54de2"
server_key_version: "kuduclusterkey@0"
I20260806 03:49:20.993952 29739 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988004637995-12279-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988004637995-12279-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988004637995-12279-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:49:21.006417 29739 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:49:21.008522 29739 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:49:21.008668 29739 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:49:21.013722 29739 rpc_server.cc:307] RPC server started. Bound to: 127.11.253.254:35823
I20260806 03:49:21.013821 29826 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.253.254:35823 every 8 connection(s)
I20260806 03:49:21.014216 29739 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988004637995-12279-0/minicluster-data/master-0/data/info.pb
I20260806 03:49:21.015023 29829 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:49:21.017515 29829 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f648b4f531da41c9a52f2a7b07c56fe3: Bootstrap starting.
I20260806 03:49:21.017618 12279 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskToVWRe/build/release/bin/kudu as pid 29739
I20260806 03:49:21.017719 12279 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988004637995-12279-0/minicluster-data/master-0/wal/instance
I20260806 03:49:21.017966 12279 external_mini_cluster.cc:1428] Setting key 0142c9e56e57567272ba64c1b23a9020
I20260806 03:49:21.018311 29829 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f648b4f531da41c9a52f2a7b07c56fe3: Neither blocks nor log segments found. Creating new log.
I20260806 03:49:21.018731 29829 log.cc:824] T 00000000000000000000000000000000 P f648b4f531da41c9a52f2a7b07c56fe3: Log is configured to *not* fsync() on all Append() calls
I20260806 03:49:21.019573 29829 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f648b4f531da41c9a52f2a7b07c56fe3: No bootstrap required, opened a new log
I20260806 03:49:21.021503 29829 raft_consensus.cc:348] T 00000000000000000000000000000000 P f648b4f531da41c9a52f2a7b07c56fe3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f648b4f531da41c9a52f2a7b07c56fe3" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 35823 } }
I20260806 03:49:21.021677 29829 raft_consensus.cc:374] T 00000000000000000000000000000000 P f648b4f531da41c9a52f2a7b07c56fe3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:49:21.021713 29829 raft_consensus.cc:729] T 00000000000000000000000000000000 P f648b4f531da41c9a52f2a7b07c56fe3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f648b4f531da41c9a52f2a7b07c56fe3, State: Initialized, Role: FOLLOWER
I20260806 03:49:21.021903 29829 consensus_queue.cc:260] T 00000000000000000000000000000000 P f648b4f531da41c9a52f2a7b07c56fe3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f648b4f531da41c9a52f2a7b07c56fe3" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 35823 } }
I20260806 03:49:21.021984 29829 raft_consensus.cc:388] T 00000000000000000000000000000000 P f648b4f531da41c9a52f2a7b07c56fe3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 03:49:21.022012 29829 raft_consensus.cc:482] T 00000000000000000000000000000000 P f648b4f531da41c9a52f2a7b07c56fe3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 03:49:21.022073 29829 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f648b4f531da41c9a52f2a7b07c56fe3 [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:49:21.022716 29829 raft_consensus.cc:504] T 00000000000000000000000000000000 P f648b4f531da41c9a52f2a7b07c56fe3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f648b4f531da41c9a52f2a7b07c56fe3" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 35823 } }
I20260806 03:49:21.022857 29829 leader_election.cc:302] T 00000000000000000000000000000000 P f648b4f531da41c9a52f2a7b07c56fe3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f648b4f531da41c9a52f2a7b07c56fe3; no voters:
I20260806 03:49:21.023085 29829 leader_election.cc:288] T 00000000000000000000000000000000 P f648b4f531da41c9a52f2a7b07c56fe3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 03:49:21.023115 29836 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f648b4f531da41c9a52f2a7b07c56fe3 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 03:49:21.023356 29836 raft_consensus.cc:686] T 00000000000000000000000000000000 P f648b4f531da41c9a52f2a7b07c56fe3 [term 1 LEADER]: Becoming Leader. State: Replica: f648b4f531da41c9a52f2a7b07c56fe3, State: Running, Role: LEADER
I20260806 03:49:21.023444 29829 sys_catalog.cc:564] T 00000000000000000000000000000000 P f648b4f531da41c9a52f2a7b07c56fe3 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 03:49:21.023494 29836 consensus_queue.cc:237] T 00000000000000000000000000000000 P f648b4f531da41c9a52f2a7b07c56fe3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f648b4f531da41c9a52f2a7b07c56fe3" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 35823 } }
I20260806 03:49:21.023900 29836 sys_catalog.cc:455] T 00000000000000000000000000000000 P f648b4f531da41c9a52f2a7b07c56fe3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f648b4f531da41c9a52f2a7b07c56fe3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f648b4f531da41c9a52f2a7b07c56fe3" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 35823 } } }
I20260806 03:49:21.024067 29836 sys_catalog.cc:458] T 00000000000000000000000000000000 P f648b4f531da41c9a52f2a7b07c56fe3 [sys.catalog]: This master's current role is: LEADER
I20260806 03:49:21.024263 29836 sys_catalog.cc:455] T 00000000000000000000000000000000 P f648b4f531da41c9a52f2a7b07c56fe3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f648b4f531da41c9a52f2a7b07c56fe3. Latest consensus state: current_term: 1 leader_uuid: "f648b4f531da41c9a52f2a7b07c56fe3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f648b4f531da41c9a52f2a7b07c56fe3" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 35823 } } }
I20260806 03:49:21.024380 29836 sys_catalog.cc:458] T 00000000000000000000000000000000 P f648b4f531da41c9a52f2a7b07c56fe3 [sys.catalog]: This master's current role is: LEADER
I20260806 03:49:21.024572 29853 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 03:49:21.024993 29829 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988004637995-12279-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260806 03:49:21.025139 29853 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 06 03:49:21 dist-test-slave-6ntk krb5kdc[29119](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988160, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.253.254@KRBTEST.COM
I20260806 03:49:21.033573 29833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:49:21.018562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33814 (local address 127.11.253.254:35823)
0806 03:49:21.018743 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0806 03:49:21.018754 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 03:49:21.018787 (+ 33us) server_negotiation.cc:407] Connection header received
0806 03:49:21.019336 (+ 549us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:49:21.019354 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:49:21.019420 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:49:21.019575 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:49:21.020384 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:49:21.022247 (+ 1863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:49:21.023334 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:49:21.023979 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:49:21.024084 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:49:21.027525 (+ 3441us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:49:21.027580 (+ 55us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:49:21.027581 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:49:21.027678 (+ 97us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:49:21.030838 (+ 3160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:49:21.031263 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:49:21.031266 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:49:21.031267 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:49:21.031379 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:49:21.031586 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:49:21.031588 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:49:21.031588 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:49:21.032001 (+ 413us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:49:21.032189 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0806 03:49:21.032749 (+ 560us) server_negotiation.cc:299] Negotiation successful
0806 03:49:21.033186 (+ 437us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":75,"threads_started":1}
I20260806 03:49:21.042527 29853 catalog_manager.cc:1347] Generated new cluster ID: 848a245868a446f48d436b2dc5b60088
I20260806 03:49:21.042591 29853 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 03:49:21.058727 29853 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 03:49:21.059589 29853 catalog_manager.cc:1504] Loading token signing keys...
I20260806 03:49:21.063681 29853 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f648b4f531da41c9a52f2a7b07c56fe3: Generated new TSK 0
I20260806 03:49:21.063959 29853 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.253.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.253.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.253.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.193.keytab.
Entry for principal kudu/127.11.253.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.193.keytab.
I20260806 03:49:21.849913 12279 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskToVWRe/build/release/bin/kudu
/tmp/dist-test-taskToVWRe/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988004637995-12279-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988004637995-12279-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988004637995-12279-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988004637995-12279-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.11.253.212:34205/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.253.193:0
--local_ip_for_outbound_sockets=127.11.253.193
--webserver_interface=127.11.253.193
--webserver_port=0
--tserver_master_addrs=127.11.253.254:35823
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.193.keytab
--principal=kudu/127.11.253.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.253.212:38447
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988004637995-12279-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988004637995-12279-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988004637995-12279-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:49:21.978423 29897 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:49:21.978765 29897 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:49:21.978816 29897 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:49:21.980500 29897 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:49:21.980571 29897 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:49:21.980611 29897 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.253.193
I20260806 03:49:21.982810 29897 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.253.212:38447
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988004637995-12279-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988004637995-12279-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.11.253.212:34205/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.193.keytab
--principal=kudu/127.11.253.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.253.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988004637995-12279-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.253.193
--webserver_port=0
--tserver_master_addrs=127.11.253.254:35823
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29897
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.253.193
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988004637995-12279-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:49:21.983080 29897 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:49:21.983357 29897 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:49:21.989478 29903 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:49:21.989589 29906 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:49:21.989656 29904 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:49:21.990163 29897 server_base.cc:1034] running on GCE node
I20260806 03:49:21.990391 29897 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:49:21.990694 29897 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:49:21.991885 29897 hybrid_clock.cc:648] HybridClock initialized: now 1785988161991862 us; error 49 us; skew 500 ppm
Aug 06 03:49:21 dist-test-slave-6ntk krb5kdc[29119](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988161, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.253.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:49:21.995253 29897 init.cc:377] Logged in from keytab as kudu/127.11.253.193@KRBTEST.COM (short username kudu)
I20260806 03:49:21.995735 29897 webserver.cc:460] Webserver started at http://127.11.253.193:37059/ using document root <none> and password file <none>
I20260806 03:49:21.996028 29897 fs_manager.cc:359] Metadata directory not provided
I20260806 03:49:21.996117 29897 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:49:21.996335 29897 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 03:49:21 dist-test-slave-6ntk krb5kdc[29119](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988161, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.253.193@KRBTEST.COM for HTTP/127.11.253.212@KRBTEST.COM
I20260806 03:49:22.031883 29897 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988004637995-12279-0/minicluster-data/ts-0/data/instance:
uuid: "f19b134ef2dd4583a52f4e7168096c40"
format_stamp: "Formatted at 2026-08-06 03:49:22 on dist-test-slave-6ntk"
server_key: "a2f176dd9a3f9cc324aad0a776030acb"
server_key_iv: "779a217f60b778ae62564b498e80cbc0"
server_key_version: "kuduclusterkey@0"
I20260806 03:49:22.032380 29897 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988004637995-12279-0/minicluster-data/ts-0/wal/instance:
uuid: "f19b134ef2dd4583a52f4e7168096c40"
format_stamp: "Formatted at 2026-08-06 03:49:22 on dist-test-slave-6ntk"
server_key: "a2f176dd9a3f9cc324aad0a776030acb"
server_key_iv: "779a217f60b778ae62564b498e80cbc0"
server_key_version: "kuduclusterkey@0"
I20260806 03:49:22.035785 29897 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.003s
I20260806 03:49:22.051854 29917 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:49:22.052136 29897 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 03:49:22.052241 29897 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988004637995-12279-0/minicluster-data/ts-0/data,/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988004637995-12279-0/minicluster-data/ts-0/wal
uuid: "f19b134ef2dd4583a52f4e7168096c40"
format_stamp: "Formatted at 2026-08-06 03:49:22 on dist-test-slave-6ntk"
server_key: "a2f176dd9a3f9cc324aad0a776030acb"
server_key_iv: "779a217f60b778ae62564b498e80cbc0"
server_key_version: "kuduclusterkey@0"
I20260806 03:49:22.052335 29897 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988004637995-12279-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988004637995-12279-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988004637995-12279-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:49:22.083065 29897 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:49:22.085728 29897 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:49:22.086017 29897 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:49:22.086264 29897 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 03:49:22.086684 29897 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 03:49:22.086745 29897 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 03:49:22.086822 29897 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 03:49:22.086853 29897 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 03:49:22.094705 29897 rpc_server.cc:307] RPC server started. Bound to: 127.11.253.193:45125
I20260806 03:49:22.094779 30083 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.253.193:45125 every 8 connection(s)
I20260806 03:49:22.095189 29897 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988004637995-12279-0/minicluster-data/ts-0/data/info.pb
Aug 06 03:49:22 dist-test-slave-6ntk krb5kdc[29119](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988161, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.253.193@KRBTEST.COM for kudu/127.11.253.254@KRBTEST.COM
I20260806 03:49:22.102885 12279 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskToVWRe/build/release/bin/kudu as pid 29897
I20260806 03:49:22.103005 12279 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988004637995-12279-0/minicluster-data/ts-0/wal/instance
I20260806 03:49:22.103240 12279 external_mini_cluster.cc:1428] Setting key 88db5cf7b015b6e90e80fa8d5c2920e1
I20260806 03:49:22.107628 30088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:49:22.096060 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:35823 (local address 127.11.253.193:57173)
0806 03:49:22.096310 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0806 03:49:22.096343 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0806 03:49:22.096877 (+ 534us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:49:22.097143 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:49:22.097148 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:49:22.097351 (+ 203us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:49:22.097549 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:49:22.097554 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:49:22.098497 (+ 943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:49:22.098498 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:49:22.099932 (+ 1434us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:49:22.099936 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:49:22.100387 (+ 451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:49:22.100388 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:49:22.100562 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:49:22.101182 (+ 620us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:49:22.101193 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:49:22.102853 (+ 1660us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:49:22.105835 (+ 2982us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:49:22.105844 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:49:22.105846 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:49:22.106194 (+ 348us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:49:22.106446 (+ 252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:49:22.106447 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:49:22.106447 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:49:22.106624 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:49:22.106953 (+ 329us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:49:22.106960 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:49:22.107125 (+ 165us) client_negotiation.cc:769] Sending connection context
0806 03:49:22.107231 (+ 106us) client_negotiation.cc:240] Negotiation successful
0806 03:49:22.107389 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":85,"threads_started":1}
I20260806 03:49:22.108191 30087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:49:22.096069 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.253.193:57173 (local address 127.11.253.254:35823)
0806 03:49:22.096182 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0806 03:49:22.096185 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 03:49:22.096438 (+ 253us) server_negotiation.cc:407] Connection header received
0806 03:49:22.096989 (+ 551us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:49:22.096991 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:49:22.097006 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:49:22.097074 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:49:22.097675 (+ 601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:49:22.098395 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:49:22.100030 (+ 1635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:49:22.100267 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:49:22.100363 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:49:22.103085 (+ 2722us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:49:22.103088 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:49:22.103088 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:49:22.103097 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:49:22.105703 (+ 2606us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:49:22.106283 (+ 580us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:49:22.106284 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:49:22.106285 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:49:22.106372 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:49:22.106705 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:49:22.106706 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:49:22.106707 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:49:22.106861 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:49:22.106926 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0806 03:49:22.107996 (+ 1070us) server_negotiation.cc:299] Negotiation successful
0806 03:49:22.108087 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88,"thread_start_us":52,"threads_started":1}
I20260806 03:49:22.108987 30084 heartbeater.cc:347] Connected to a master server at 127.11.253.254:35823
I20260806 03:49:22.109079 30084 heartbeater.cc:464] Registering TS with master...
I20260806 03:49:22.109272 30084 heartbeater.cc:511] Master 127.11.253.254:35823 requested a full tablet report, sending...
I20260806 03:49:22.109907 29781 ts_manager.cc:194] Registered new tserver with Master: f19b134ef2dd4583a52f4e7168096c40 (127.11.253.193:45125)
I20260806 03:49:22.110749 29781 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.253.193@KRBTEST.COM'} at 127.11.253.193:57173
WARNING: no policy specified for kudu/127.11.253.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.253.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.253.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.194.keytab.
Entry for principal kudu/127.11.253.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.194.keytab.
I20260806 03:49:22.142011 12279 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskToVWRe/build/release/bin/kudu
/tmp/dist-test-taskToVWRe/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988004637995-12279-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988004637995-12279-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988004637995-12279-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988004637995-12279-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.11.253.212:34205/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.253.194:0
--local_ip_for_outbound_sockets=127.11.253.194
--webserver_interface=127.11.253.194
--webserver_port=0
--tserver_master_addrs=127.11.253.254:35823
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.194.keytab
--principal=kudu/127.11.253.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.253.212:38447
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988004637995-12279-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988004637995-12279-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988004637995-12279-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 03:49:22 dist-test-slave-6ntk krb5kdc[29119](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1785988162, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.253.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260806 03:49:22.268496 30091 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:49:22.268800 30091 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:49:22.268862 30091 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:49:22.270491 30091 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:49:22.270581 30091 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:49:22.270637 30091 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.253.194
I20260806 03:49:22.272814 30091 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.253.212:38447
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988004637995-12279-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988004637995-12279-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.11.253.212:34205/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.194.keytab
--principal=kudu/127.11.253.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.253.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988004637995-12279-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.253.194
--webserver_port=0
--tserver_master_addrs=127.11.253.254:35823
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30091
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.253.194
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988004637995-12279-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:49:22.273221 30091 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:49:22.273551 30091 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 03:49:22.282601 30091 server_base.cc:1034] running on GCE node
W20260806 03:49:22.282858 30105 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:49:22.283053 30102 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:49:22.284584 30103 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:49:22.284883 30091 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:49:22.285199 30091 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:49:22.286396 30091 hybrid_clock.cc:648] HybridClock initialized: now 1785988162286372 us; error 40 us; skew 500 ppm
Aug 06 03:49:22 dist-test-slave-6ntk krb5kdc[29119](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988162, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.253.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:49:22.289556 30091 init.cc:377] Logged in from keytab as kudu/127.11.253.194@KRBTEST.COM (short username kudu)
I20260806 03:49:22.290143 30091 webserver.cc:460] Webserver started at http://127.11.253.194:44039/ using document root <none> and password file <none>
I20260806 03:49:22.290402 30091 fs_manager.cc:359] Metadata directory not provided
I20260806 03:49:22.290473 30091 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:49:22.290619 30091 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 03:49:22 dist-test-slave-6ntk krb5kdc[29119](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988162, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.253.194@KRBTEST.COM for HTTP/127.11.253.212@KRBTEST.COM
I20260806 03:49:22.308236 30091 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988004637995-12279-0/minicluster-data/ts-1/data/instance:
uuid: "49f681b837fa45909ee3a260437772a6"
format_stamp: "Formatted at 2026-08-06 03:49:22 on dist-test-slave-6ntk"
server_key: "45cfe90dfb7ed3be6b60501ee91055af"
server_key_iv: "b05908157d2022574759f5de0a8bc677"
server_key_version: "kuduclusterkey@0"
I20260806 03:49:22.308794 30091 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988004637995-12279-0/minicluster-data/ts-1/wal/instance:
uuid: "49f681b837fa45909ee3a260437772a6"
format_stamp: "Formatted at 2026-08-06 03:49:22 on dist-test-slave-6ntk"
server_key: "45cfe90dfb7ed3be6b60501ee91055af"
server_key_iv: "b05908157d2022574759f5de0a8bc677"
server_key_version: "kuduclusterkey@0"
I20260806 03:49:22.311024 30091 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.004s sys 0.000s
I20260806 03:49:22.326324 30114 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:49:22.334082 30091 fs_manager.cc:725] Time spent opening block manager: real 0.011s user 0.001s sys 0.000s
I20260806 03:49:22.334245 30091 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988004637995-12279-0/minicluster-data/ts-1/data,/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988004637995-12279-0/minicluster-data/ts-1/wal
uuid: "49f681b837fa45909ee3a260437772a6"
format_stamp: "Formatted at 2026-08-06 03:49:22 on dist-test-slave-6ntk"
server_key: "45cfe90dfb7ed3be6b60501ee91055af"
server_key_iv: "b05908157d2022574759f5de0a8bc677"
server_key_version: "kuduclusterkey@0"
I20260806 03:49:22.334370 30091 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988004637995-12279-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988004637995-12279-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988004637995-12279-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:49:22.366794 30091 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:49:22.370178 30091 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:49:22.370456 30091 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:49:22.370767 30091 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 03:49:22.371276 30091 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 03:49:22.371362 30091 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 03:49:22.371464 30091 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 03:49:22.371526 30091 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 03:49:22.379091 30091 rpc_server.cc:307] RPC server started. Bound to: 127.11.253.194:34161
I20260806 03:49:22.379639 30091 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988004637995-12279-0/minicluster-data/ts-1/data/info.pb
I20260806 03:49:22.382035 30279 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.253.194:34161 every 8 connection(s)
I20260806 03:49:22.384968 12279 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskToVWRe/build/release/bin/kudu as pid 30091
I20260806 03:49:22.385093 12279 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988004637995-12279-0/minicluster-data/ts-1/wal/instance
I20260806 03:49:22.385329 12279 external_mini_cluster.cc:1428] Setting key 6fe5c327d154f994414a7a34c33a7f85
Aug 06 03:49:22 dist-test-slave-6ntk krb5kdc[29119](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988162, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.253.194@KRBTEST.COM for kudu/127.11.253.254@KRBTEST.COM
I20260806 03:49:22.398921 30087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:49:22.380963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.253.194:60683 (local address 127.11.253.254:35823)
0806 03:49:22.381014 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0806 03:49:22.381016 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:49:22.381097 (+ 81us) server_negotiation.cc:407] Connection header received
0806 03:49:22.386019 (+ 4922us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:49:22.386021 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:49:22.386037 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:49:22.386107 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:49:22.387162 (+ 1055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:49:22.387943 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:49:22.389541 (+ 1598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:49:22.389859 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:49:22.389925 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:49:22.392659 (+ 2734us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:49:22.392661 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:49:22.392661 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:49:22.392669 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:49:22.394457 (+ 1788us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:49:22.397534 (+ 3077us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:49:22.397535 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:49:22.397535 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:49:22.397605 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:49:22.397966 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:49:22.397967 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:49:22.397968 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:49:22.398132 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:49:22.398179 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0806 03:49:22.398709 (+ 530us) server_negotiation.cc:299] Negotiation successful
0806 03:49:22.398816 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 03:49:22.399736 30284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:49:22.380650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:35823 (local address 127.11.253.194:60683)
0806 03:49:22.380937 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0806 03:49:22.380994 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0806 03:49:22.382172 (+ 1178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:49:22.386437 (+ 4265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:49:22.386443 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:49:22.386738 (+ 295us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:49:22.387025 (+ 287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:49:22.387032 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:49:22.388074 (+ 1042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:49:22.388075 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:49:22.389436 (+ 1361us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:49:22.389439 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:49:22.390019 (+ 580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:49:22.390020 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:49:22.390163 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:49:22.390832 (+ 669us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:49:22.390843 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:49:22.392541 (+ 1698us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:49:22.394574 (+ 2033us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:49:22.394582 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:49:22.394584 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:49:22.394895 (+ 311us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:49:22.397683 (+ 2788us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:49:22.397684 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:49:22.397684 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:49:22.397845 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:49:22.398248 (+ 403us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:49:22.398257 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:49:22.398510 (+ 253us) client_negotiation.cc:769] Sending connection context
0806 03:49:22.399290 (+ 780us) client_negotiation.cc:240] Negotiation successful
0806 03:49:22.399477 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":226,"thread_start_us":85,"threads_started":1}
I20260806 03:49:22.400413 30280 heartbeater.cc:347] Connected to a master server at 127.11.253.254:35823
I20260806 03:49:22.400511 30280 heartbeater.cc:464] Registering TS with master...
I20260806 03:49:22.400703 30280 heartbeater.cc:511] Master 127.11.253.254:35823 requested a full tablet report, sending...
I20260806 03:49:22.401171 29781 ts_manager.cc:194] Registered new tserver with Master: 49f681b837fa45909ee3a260437772a6 (127.11.253.194:34161)
I20260806 03:49:22.401659 29781 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.253.194@KRBTEST.COM'} at 127.11.253.194:60683
WARNING: no policy specified for kudu/127.11.253.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.253.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.253.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.195.keytab.
Entry for principal kudu/127.11.253.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.195.keytab.
I20260806 03:49:22.437968 12279 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskToVWRe/build/release/bin/kudu
/tmp/dist-test-taskToVWRe/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988004637995-12279-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988004637995-12279-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988004637995-12279-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988004637995-12279-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.11.253.212:34205/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.253.195:0
--local_ip_for_outbound_sockets=127.11.253.195
--webserver_interface=127.11.253.195
--webserver_port=0
--tserver_master_addrs=127.11.253.254:35823
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.195.keytab
--principal=kudu/127.11.253.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.253.212:38447
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988004637995-12279-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988004637995-12279-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988004637995-12279-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:49:22.568341 30287 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:49:22.568714 30287 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:49:22.568832 30287 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:49:22.570644 30287 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:49:22.570779 30287 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:49:22.570868 30287 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.253.195
I20260806 03:49:22.573146 30287 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.253.212:38447
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988004637995-12279-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988004637995-12279-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.11.253.212:34205/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.195.keytab
--principal=kudu/127.11.253.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.253.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988004637995-12279-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.253.195
--webserver_port=0
--tserver_master_addrs=127.11.253.254:35823
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30287
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.253.195
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988004637995-12279-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:49:22.573626 30287 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:49:22.574019 30287 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:49:22.580933 30297 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:49:22.581198 30295 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:49:22.581225 30294 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:49:22.581581 30287 server_base.cc:1034] running on GCE node
I20260806 03:49:22.581806 30287 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:49:22.582103 30287 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:49:22.583283 30287 hybrid_clock.cc:648] HybridClock initialized: now 1785988162583262 us; error 34 us; skew 500 ppm
Aug 06 03:49:22 dist-test-slave-6ntk krb5kdc[29119](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988162, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.253.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:49:22.586598 30287 init.cc:377] Logged in from keytab as kudu/127.11.253.195@KRBTEST.COM (short username kudu)
I20260806 03:49:22.587145 30287 webserver.cc:460] Webserver started at http://127.11.253.195:34965/ using document root <none> and password file <none>
I20260806 03:49:22.587466 30287 fs_manager.cc:359] Metadata directory not provided
I20260806 03:49:22.587554 30287 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:49:22.587708 30287 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 03:49:22 dist-test-slave-6ntk krb5kdc[29119](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988162, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.253.195@KRBTEST.COM for HTTP/127.11.253.212@KRBTEST.COM
I20260806 03:49:22.602520 30287 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988004637995-12279-0/minicluster-data/ts-2/data/instance:
uuid: "63b63eea39d4499fac09468b36792700"
format_stamp: "Formatted at 2026-08-06 03:49:22 on dist-test-slave-6ntk"
server_key: "c082b22a2cc035c21684e6a7a7bd692e"
server_key_iv: "b7f3e8d3b6fee7136ec050b01b4e1960"
server_key_version: "kuduclusterkey@0"
I20260806 03:49:22.602977 30287 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988004637995-12279-0/minicluster-data/ts-2/wal/instance:
uuid: "63b63eea39d4499fac09468b36792700"
format_stamp: "Formatted at 2026-08-06 03:49:22 on dist-test-slave-6ntk"
server_key: "c082b22a2cc035c21684e6a7a7bd692e"
server_key_iv: "b7f3e8d3b6fee7136ec050b01b4e1960"
server_key_version: "kuduclusterkey@0"
I20260806 03:49:22.607002 30287 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260806 03:49:22.638036 30306 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:49:22.638374 30287 fs_manager.cc:725] Time spent opening block manager: real 0.018s user 0.001s sys 0.000s
I20260806 03:49:22.638515 30287 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988004637995-12279-0/minicluster-data/ts-2/data,/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988004637995-12279-0/minicluster-data/ts-2/wal
uuid: "63b63eea39d4499fac09468b36792700"
format_stamp: "Formatted at 2026-08-06 03:49:22 on dist-test-slave-6ntk"
server_key: "c082b22a2cc035c21684e6a7a7bd692e"
server_key_iv: "b7f3e8d3b6fee7136ec050b01b4e1960"
server_key_version: "kuduclusterkey@0"
I20260806 03:49:22.638628 30287 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988004637995-12279-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988004637995-12279-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988004637995-12279-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:49:22.677542 30287 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:49:22.680635 30287 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:49:22.680910 30287 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:49:22.681187 30287 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 03:49:22.681653 30287 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 03:49:22.681769 30287 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 03:49:22.681851 30287 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 03:49:22.681937 30287 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 03:49:22.689787 30287 rpc_server.cc:307] RPC server started. Bound to: 127.11.253.195:45445
I20260806 03:49:22.690970 30287 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988004637995-12279-0/minicluster-data/ts-2/data/info.pb
I20260806 03:49:22.693252 30474 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.253.195:45445 every 8 connection(s)
I20260806 03:49:22.699370 12279 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskToVWRe/build/release/bin/kudu as pid 30287
I20260806 03:49:22.699493 12279 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988004637995-12279-0/minicluster-data/ts-2/wal/instance
I20260806 03:49:22.699685 12279 external_mini_cluster.cc:1428] Setting key eaa8980006ea1fe83caecc8d8d974304
Aug 06 03:49:22 dist-test-slave-6ntk krb5kdc[29119](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988162, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.253.195@KRBTEST.COM for kudu/127.11.253.254@KRBTEST.COM
I20260806 03:49:22.708491 30087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:49:22.693967 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.253.195:51513 (local address 127.11.253.254:35823)
0806 03:49:22.694030 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0806 03:49:22.694032 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:49:22.697892 (+ 3860us) server_negotiation.cc:407] Connection header received
0806 03:49:22.698676 (+ 784us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:49:22.698678 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:49:22.698693 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:49:22.698724 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:49:22.699677 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:49:22.700430 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:49:22.702112 (+ 1682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:49:22.702465 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:49:22.702508 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:49:22.705185 (+ 2677us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:49:22.705186 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:49:22.705187 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:49:22.705195 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:49:22.706901 (+ 1706us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:49:22.707399 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:49:22.707401 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:49:22.707401 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:49:22.707462 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:49:22.707731 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:49:22.707732 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:49:22.707732 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:49:22.707862 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:49:22.707904 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0806 03:49:22.708296 (+ 392us) server_negotiation.cc:299] Negotiation successful
0806 03:49:22.708396 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260806 03:49:22.709014 30477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:49:22.693965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:35823 (local address 127.11.253.195:51513)
0806 03:49:22.697675 (+ 3710us) negotiation.cc:107] Waiting for socket to connect
0806 03:49:22.697749 (+ 74us) client_negotiation.cc:174] Beginning negotiation
0806 03:49:22.698512 (+ 763us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:49:22.698940 (+ 428us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:49:22.698944 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:49:22.699248 (+ 304us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:49:22.699566 (+ 318us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:49:22.699573 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:49:22.700678 (+ 1105us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:49:22.700678 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:49:22.702017 (+ 1339us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:49:22.702020 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:49:22.702614 (+ 594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:49:22.702614 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:49:22.702780 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:49:22.703455 (+ 675us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:49:22.703466 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:49:22.705073 (+ 1607us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:49:22.706992 (+ 1919us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:49:22.707000 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:49:22.707003 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:49:22.707300 (+ 297us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:49:22.707532 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:49:22.707533 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:49:22.707533 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:49:22.707667 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:49:22.707955 (+ 288us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:49:22.707961 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:49:22.708135 (+ 174us) client_negotiation.cc:769] Sending connection context
0806 03:49:22.708603 (+ 468us) client_negotiation.cc:240] Negotiation successful
0806 03:49:22.708777 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3643,"thread_start_us":115,"threads_started":1}
I20260806 03:49:22.709512 30475 heartbeater.cc:347] Connected to a master server at 127.11.253.254:35823
I20260806 03:49:22.709619 30475 heartbeater.cc:464] Registering TS with master...
I20260806 03:49:22.709820 30475 heartbeater.cc:511] Master 127.11.253.254:35823 requested a full tablet report, sending...
I20260806 03:49:22.710208 29781 ts_manager.cc:194] Registered new tserver with Master: 63b63eea39d4499fac09468b36792700 (127.11.253.195:45445)
I20260806 03:49:22.710695 29781 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.253.195@KRBTEST.COM'} at 127.11.253.195:51513
I20260806 03:49:22.711887 12279 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 03:49:22.712702 12279 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskToVWRe/build/release/bin/kudu with pid 29897
I20260806 03:49:22.745620 12279 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskToVWRe/build/release/bin/kudu with pid 30091
I20260806 03:49:22.758083 12279 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskToVWRe/build/release/bin/kudu with pid 30287
I20260806 03:49:22.785063 12279 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskToVWRe/build/release/bin/kudu with pid 29739
I20260806 03:49:22.796286 12279 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskToVWRe/build/release/bin/kudu
/tmp/dist-test-taskToVWRe/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988004637995-12279-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988004637995-12279-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988004637995-12279-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988004637995-12279-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.253.254:35823
--webserver_interface=127.11.253.254
--webserver_port=40951
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.254.keytab
--principal=kudu/127.11.253.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.253.212:38447
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.253.254:35823
--ranger_config_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988004637995-12279-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.11.253.212:34205/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.11.253.212:34205/kms with env {KRB5CCNAME=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988004637995-12279-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988004637995-12279-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988004637995-12279-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:49:22.881840 30478 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:49:22.882149 30478 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:49:22.882189 30478 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:49:22.883334 30478 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 03:49:22.883409 30478 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:49:22.883437 30478 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:49:22.883468 30478 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 03:49:22.883490 30478 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 03:49:22.885089 30478 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.253.212:38447
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988004637995-12279-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988004637995-12279-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.11.253.212:34205/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.11.253.254:35823
--ranger_config_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988004637995-12279-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.254.keytab
--principal=kudu/127.11.253.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.253.254:35823
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988004637995-12279-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.253.254
--webserver_port=40951
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30478
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988004637995-12279-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:49:22.885372 30478 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:49:22.885615 30478 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:49:22.890794 30489 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:49:22.890820 30486 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:49:22.891038 30491 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:49:22.891098 30478 server_base.cc:1034] running on GCE node
I20260806 03:49:22.891522 30478 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:49:22.891896 30478 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:49:22.893090 30478 hybrid_clock.cc:648] HybridClock initialized: now 1785988162893060 us; error 42 us; skew 500 ppm
Aug 06 03:49:22 dist-test-slave-6ntk krb5kdc[29119](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988162, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.253.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:49:22.896363 30478 init.cc:377] Logged in from keytab as kudu/127.11.253.254@KRBTEST.COM (short username kudu)
I20260806 03:49:22.896952 30478 webserver.cc:460] Webserver started at http://127.11.253.254:40951/ using document root <none> and password file <none>
I20260806 03:49:22.897224 30478 fs_manager.cc:359] Metadata directory not provided
I20260806 03:49:22.897315 30478 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 06 03:49:22 dist-test-slave-6ntk krb5kdc[29119](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988162, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.253.254@KRBTEST.COM for HTTP/127.11.253.212@KRBTEST.COM
I20260806 03:49:22.944336 30478 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 03:49:22.945209 30500 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:49:22.945422 30478 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 03:49:22.945550 30478 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988004637995-12279-0/minicluster-data/master-0/data,/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988004637995-12279-0/minicluster-data/master-0/wal
uuid: "f648b4f531da41c9a52f2a7b07c56fe3"
format_stamp: "Formatted at 2026-08-06 03:49:20 on dist-test-slave-6ntk"
server_key: "2b68e3cf447d7c5858904eeb9810ba0a"
server_key_iv: "3c1446471ac9ded099ffc3d13af54de2"
server_key_version: "kuduclusterkey@0"
I20260806 03:49:22.946009 30478 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988004637995-12279-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988004637995-12279-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988004637995-12279-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:49:22.961444 30478 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:49:22.963689 30478 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:49:22.963851 30478 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:49:22.968413 30478 rpc_server.cc:307] RPC server started. Bound to: 127.11.253.254:35823
I20260806 03:49:22.968461 30577 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.253.254:35823 every 8 connection(s)
I20260806 03:49:22.968830 30478 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988004637995-12279-0/minicluster-data/master-0/data/info.pb
I20260806 03:49:22.970261 30579 sys_catalog.cc:263] Verifying existing consensus state
I20260806 03:49:22.970937 30579 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f648b4f531da41c9a52f2a7b07c56fe3: Bootstrap starting.
I20260806 03:49:22.972901 12279 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskToVWRe/build/release/bin/kudu as pid 30478
I20260806 03:49:22.973218 12279 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskToVWRe/build/release/bin/kudu
/tmp/dist-test-taskToVWRe/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988004637995-12279-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988004637995-12279-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988004637995-12279-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988004637995-12279-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.11.253.212:34205/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.253.193:45125
--local_ip_for_outbound_sockets=127.11.253.193
--tserver_master_addrs=127.11.253.254:35823
--webserver_port=37059
--webserver_interface=127.11.253.193
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.193.keytab
--principal=kudu/127.11.253.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.253.212:38447
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.11.253.212:34205/kms with env {KRB5CCNAME=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988004637995-12279-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988004637995-12279-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988004637995-12279-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 03:49:22.979734 30579 log.cc:824] T 00000000000000000000000000000000 P f648b4f531da41c9a52f2a7b07c56fe3: Log is configured to *not* fsync() on all Append() calls
I20260806 03:49:22.981597 30579 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f648b4f531da41c9a52f2a7b07c56fe3: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 03:49:22.981980 30579 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f648b4f531da41c9a52f2a7b07c56fe3: Bootstrap complete.
I20260806 03:49:22.984431 30579 raft_consensus.cc:348] T 00000000000000000000000000000000 P f648b4f531da41c9a52f2a7b07c56fe3 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f648b4f531da41c9a52f2a7b07c56fe3" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 35823 } }
I20260806 03:49:22.984689 30579 raft_consensus.cc:729] T 00000000000000000000000000000000 P f648b4f531da41c9a52f2a7b07c56fe3 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: f648b4f531da41c9a52f2a7b07c56fe3, State: Initialized, Role: FOLLOWER
I20260806 03:49:22.984918 30579 consensus_queue.cc:260] T 00000000000000000000000000000000 P f648b4f531da41c9a52f2a7b07c56fe3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f648b4f531da41c9a52f2a7b07c56fe3" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 35823 } }
I20260806 03:49:22.985021 30579 raft_consensus.cc:388] T 00000000000000000000000000000000 P f648b4f531da41c9a52f2a7b07c56fe3 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 03:49:22.985090 30579 raft_consensus.cc:482] T 00000000000000000000000000000000 P f648b4f531da41c9a52f2a7b07c56fe3 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 03:49:22.985168 30579 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f648b4f531da41c9a52f2a7b07c56fe3 [term 1 FOLLOWER]: Advancing to term 2
I20260806 03:49:22.986306 30579 raft_consensus.cc:504] T 00000000000000000000000000000000 P f648b4f531da41c9a52f2a7b07c56fe3 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f648b4f531da41c9a52f2a7b07c56fe3" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 35823 } }
I20260806 03:49:22.986433 30579 leader_election.cc:302] T 00000000000000000000000000000000 P f648b4f531da41c9a52f2a7b07c56fe3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f648b4f531da41c9a52f2a7b07c56fe3; no voters:
I20260806 03:49:22.986631 30579 leader_election.cc:288] T 00000000000000000000000000000000 P f648b4f531da41c9a52f2a7b07c56fe3 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260806 03:49:22.986666 30586 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f648b4f531da41c9a52f2a7b07c56fe3 [term 2 FOLLOWER]: Leader election won for term 2
I20260806 03:49:22.986882 30586 raft_consensus.cc:686] T 00000000000000000000000000000000 P f648b4f531da41c9a52f2a7b07c56fe3 [term 2 LEADER]: Becoming Leader. State: Replica: f648b4f531da41c9a52f2a7b07c56fe3, State: Running, Role: LEADER
I20260806 03:49:22.987003 30579 sys_catalog.cc:564] T 00000000000000000000000000000000 P f648b4f531da41c9a52f2a7b07c56fe3 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 03:49:22.987030 30586 consensus_queue.cc:237] T 00000000000000000000000000000000 P f648b4f531da41c9a52f2a7b07c56fe3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f648b4f531da41c9a52f2a7b07c56fe3" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 35823 } }
I20260806 03:49:22.987368 30586 sys_catalog.cc:455] T 00000000000000000000000000000000 P f648b4f531da41c9a52f2a7b07c56fe3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "f648b4f531da41c9a52f2a7b07c56fe3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f648b4f531da41c9a52f2a7b07c56fe3" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 35823 } } }
I20260806 03:49:22.987597 30586 sys_catalog.cc:458] T 00000000000000000000000000000000 P f648b4f531da41c9a52f2a7b07c56fe3 [sys.catalog]: This master's current role is: LEADER
I20260806 03:49:22.987795 30586 sys_catalog.cc:455] T 00000000000000000000000000000000 P f648b4f531da41c9a52f2a7b07c56fe3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f648b4f531da41c9a52f2a7b07c56fe3. Latest consensus state: current_term: 2 leader_uuid: "f648b4f531da41c9a52f2a7b07c56fe3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f648b4f531da41c9a52f2a7b07c56fe3" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 35823 } } }
I20260806 03:49:22.987900 30586 sys_catalog.cc:458] T 00000000000000000000000000000000 P f648b4f531da41c9a52f2a7b07c56fe3 [sys.catalog]: This master's current role is: LEADER
I20260806 03:49:22.988049 30599 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 03:49:22.988701 30599 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 03:49:22.989089 30579 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988004637995-12279-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260806 03:49:22.989102 30599 catalog_manager.cc:1259] Loaded cluster ID: 848a245868a446f48d436b2dc5b60088
I20260806 03:49:22.989377 30599 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 03:49:22.990303 30599 catalog_manager.cc:1504] Loading token signing keys...
I20260806 03:49:22.990496 30599 catalog_manager.cc:5879] T 00000000000000000000000000000000 P f648b4f531da41c9a52f2a7b07c56fe3: Loaded TSK: 0
I20260806 03:49:22.990710 30599 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 03:49:23.077054 30581 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:49:23.077371 30581 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:49:23.077443 30581 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:49:23.078727 30581 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:49:23.078799 30581 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:49:23.078882 30581 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.253.193
I20260806 03:49:23.080718 30581 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.253.212:38447
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988004637995-12279-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988004637995-12279-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.11.253.212:34205/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.193.keytab
--principal=kudu/127.11.253.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.253.193:45125
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988004637995-12279-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.253.193
--webserver_port=37059
--tserver_master_addrs=127.11.253.254:35823
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30581
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.253.193
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988004637995-12279-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:49:23.081082 30581 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:49:23.081372 30581 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:49:23.086807 30633 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:49:23.087013 30632 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:49:23.087278 30636 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:49:23.088145 30581 server_base.cc:1034] running on GCE node
I20260806 03:49:23.088390 30581 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:49:23.088691 30581 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:49:23.092000 30581 hybrid_clock.cc:648] HybridClock initialized: now 1785988163091983 us; error 29 us; skew 500 ppm
Aug 06 03:49:23 dist-test-slave-6ntk krb5kdc[29119](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988163, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.253.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:49:23.094983 30581 init.cc:377] Logged in from keytab as kudu/127.11.253.193@KRBTEST.COM (short username kudu)
I20260806 03:49:23.095512 30581 webserver.cc:460] Webserver started at http://127.11.253.193:37059/ using document root <none> and password file <none>
I20260806 03:49:23.095750 30581 fs_manager.cc:359] Metadata directory not provided
I20260806 03:49:23.095834 30581 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 06 03:49:23 dist-test-slave-6ntk krb5kdc[29119](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988163, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.253.193@KRBTEST.COM for HTTP/127.11.253.212@KRBTEST.COM
I20260806 03:49:23.123332 30581 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 03:49:23.124166 30644 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:49:23.124341 30581 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260806 03:49:23.124447 30581 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988004637995-12279-0/minicluster-data/ts-0/data,/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988004637995-12279-0/minicluster-data/ts-0/wal
uuid: "f19b134ef2dd4583a52f4e7168096c40"
format_stamp: "Formatted at 2026-08-06 03:49:22 on dist-test-slave-6ntk"
server_key: "a2f176dd9a3f9cc324aad0a776030acb"
server_key_iv: "779a217f60b778ae62564b498e80cbc0"
server_key_version: "kuduclusterkey@0"
I20260806 03:49:23.124859 30581 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988004637995-12279-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988004637995-12279-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988004637995-12279-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:49:23.149894 30581 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:49:23.152455 30581 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:49:23.152702 30581 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:49:23.153085 30581 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 03:49:23.153537 30581 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 03:49:23.153640 30581 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 03:49:23.153746 30581 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 03:49:23.153841 30581 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 03:49:23.160967 30581 rpc_server.cc:307] RPC server started. Bound to: 127.11.253.193:45125
I20260806 03:49:23.161219 30813 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.253.193:45125 every 8 connection(s)
I20260806 03:49:23.161521 30581 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988004637995-12279-0/minicluster-data/ts-0/data/info.pb
Aug 06 03:49:23 dist-test-slave-6ntk krb5kdc[29119](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988163, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.253.193@KRBTEST.COM for kudu/127.11.253.254@KRBTEST.COM
I20260806 03:49:23.171367 12279 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskToVWRe/build/release/bin/kudu as pid 30581
I20260806 03:49:23.171706 12279 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskToVWRe/build/release/bin/kudu
/tmp/dist-test-taskToVWRe/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988004637995-12279-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988004637995-12279-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988004637995-12279-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988004637995-12279-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.11.253.212:34205/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.253.194:34161
--local_ip_for_outbound_sockets=127.11.253.194
--tserver_master_addrs=127.11.253.254:35823
--webserver_port=44039
--webserver_interface=127.11.253.194
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.194.keytab
--principal=kudu/127.11.253.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.253.212:38447
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.11.253.212:34205/kms with env {KRB5CCNAME=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988004637995-12279-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988004637995-12279-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988004637995-12279-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 03:49:23.178586 30817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:49:23.162381 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:35823 (local address 127.11.253.193:44275)
0806 03:49:23.162649 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0806 03:49:23.162694 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 03:49:23.163317 (+ 623us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:49:23.163990 (+ 673us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:49:23.163994 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:49:23.164242 (+ 248us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:49:23.164748 (+ 506us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:49:23.164755 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:49:23.166405 (+ 1650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:49:23.166407 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:49:23.167589 (+ 1182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:49:23.167592 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:49:23.168256 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:49:23.168257 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:49:23.168379 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:49:23.168999 (+ 620us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:49:23.169007 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:49:23.173577 (+ 4570us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:49:23.176140 (+ 2563us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:49:23.176147 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:49:23.176149 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:49:23.176471 (+ 322us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:49:23.176778 (+ 307us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:49:23.176779 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:49:23.176780 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:49:23.177212 (+ 432us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:49:23.177760 (+ 548us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:49:23.177766 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:49:23.177988 (+ 222us) client_negotiation.cc:769] Sending connection context
0806 03:49:23.178109 (+ 121us) client_negotiation.cc:240] Negotiation successful
0806 03:49:23.178287 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":79,"threads_started":1}
I20260806 03:49:23.179163 30818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:49:23.162476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.253.193:44275 (local address 127.11.253.254:35823)
0806 03:49:23.162674 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0806 03:49:23.162693 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0806 03:49:23.162782 (+ 89us) server_negotiation.cc:407] Connection header received
0806 03:49:23.163612 (+ 830us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:49:23.163619 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:49:23.163679 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:49:23.163812 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:49:23.164888 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:49:23.166274 (+ 1386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:49:23.167705 (+ 1431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:49:23.168131 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:49:23.168258 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:49:23.173732 (+ 5474us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:49:23.173738 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:49:23.173738 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:49:23.173751 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:49:23.176048 (+ 2297us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:49:23.176564 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:49:23.176567 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:49:23.176568 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:49:23.176696 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:49:23.177302 (+ 606us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:49:23.177303 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:49:23.177304 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:49:23.177636 (+ 332us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:49:23.177727 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0806 03:49:23.178795 (+ 1068us) server_negotiation.cc:299] Negotiation successful
0806 03:49:23.178940 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153,"thread_start_us":74,"threads_started":1}
I20260806 03:49:23.179978 30815 heartbeater.cc:347] Connected to a master server at 127.11.253.254:35823
I20260806 03:49:23.180074 30815 heartbeater.cc:464] Registering TS with master...
I20260806 03:49:23.180264 30815 heartbeater.cc:511] Master 127.11.253.254:35823 requested a full tablet report, sending...
W20260806 03:49:23.181254 30815 heartbeater.cc:644] Failed to heartbeat to 127.11.253.254:35823 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260806 03:49:23.181340 30815 heartbeater.cc:464] Registering TS with master...
I20260806 03:49:23.181406 30815 heartbeater.cc:511] Master 127.11.253.254:35823 requested a full tablet report, sending...
I20260806 03:49:23.181797 30815 heartbeater.cc:464] Registering TS with master...
I20260806 03:49:23.181871 30815 heartbeater.cc:511] Master 127.11.253.254:35823 requested a full tablet report, sending...
W20260806 03:49:23.182294 30815 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 03:49:23.285243 30821 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:49:23.285637 30821 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:49:23.285745 30821 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:49:23.287705 30821 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:49:23.287837 30821 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:49:23.287943 30821 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.253.194
I20260806 03:49:23.290254 30821 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.253.212:38447
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988004637995-12279-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988004637995-12279-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.11.253.212:34205/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.194.keytab
--principal=kudu/127.11.253.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.253.194:34161
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988004637995-12279-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.253.194
--webserver_port=44039
--tserver_master_addrs=127.11.253.254:35823
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30821
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.253.194
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988004637995-12279-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:49:23.290556 30821 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:49:23.290879 30821 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:49:23.297032 30828 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:49:23.297041 30830 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:49:23.298237 30821 server_base.cc:1034] running on GCE node
W20260806 03:49:23.298782 30827 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:49:23.299047 30821 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:49:23.299315 30821 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:49:23.302618 30821 hybrid_clock.cc:648] HybridClock initialized: now 1785988163302593 us; error 38 us; skew 500 ppm
Aug 06 03:49:23 dist-test-slave-6ntk krb5kdc[29119](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988163, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.253.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:49:23.306018 30821 init.cc:377] Logged in from keytab as kudu/127.11.253.194@KRBTEST.COM (short username kudu)
I20260806 03:49:23.306555 30821 webserver.cc:460] Webserver started at http://127.11.253.194:44039/ using document root <none> and password file <none>
I20260806 03:49:23.306814 30821 fs_manager.cc:359] Metadata directory not provided
I20260806 03:49:23.306900 30821 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 06 03:49:23 dist-test-slave-6ntk krb5kdc[29119](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988163, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.253.194@KRBTEST.COM for HTTP/127.11.253.212@KRBTEST.COM
I20260806 03:49:23.320751 30821 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 03:49:23.321807 30842 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:49:23.322074 30821 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260806 03:49:23.322165 30821 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988004637995-12279-0/minicluster-data/ts-1/data,/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988004637995-12279-0/minicluster-data/ts-1/wal
uuid: "49f681b837fa45909ee3a260437772a6"
format_stamp: "Formatted at 2026-08-06 03:49:22 on dist-test-slave-6ntk"
server_key: "45cfe90dfb7ed3be6b60501ee91055af"
server_key_iv: "b05908157d2022574759f5de0a8bc677"
server_key_version: "kuduclusterkey@0"
I20260806 03:49:23.322625 30821 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988004637995-12279-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988004637995-12279-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988004637995-12279-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:49:23.332091 30821 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:49:23.334291 30821 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:49:23.334466 30821 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:49:23.334753 30821 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 03:49:23.335175 30821 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 03:49:23.335228 30821 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 03:49:23.335260 30821 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 03:49:23.335330 30821 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 03:49:23.343097 30821 rpc_server.cc:307] RPC server started. Bound to: 127.11.253.194:34161
I20260806 03:49:23.343240 31010 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.253.194:34161 every 8 connection(s)
I20260806 03:49:23.343626 30821 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988004637995-12279-0/minicluster-data/ts-1/data/info.pb
I20260806 03:49:23.349793 12279 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskToVWRe/build/release/bin/kudu as pid 30821
I20260806 03:49:23.350407 12279 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskToVWRe/build/release/bin/kudu
/tmp/dist-test-taskToVWRe/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988004637995-12279-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988004637995-12279-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988004637995-12279-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988004637995-12279-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.11.253.212:34205/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.253.195:45445
--local_ip_for_outbound_sockets=127.11.253.195
--tserver_master_addrs=127.11.253.254:35823
--webserver_port=34965
--webserver_interface=127.11.253.195
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.195.keytab
--principal=kudu/127.11.253.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.253.212:38447
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.11.253.212:34205/kms with env {KRB5CCNAME=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988004637995-12279-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988004637995-12279-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988004637995-12279-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 03:49:23 dist-test-slave-6ntk krb5kdc[29119](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988163, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.253.194@KRBTEST.COM for kudu/127.11.253.254@KRBTEST.COM
I20260806 03:49:23.355949 30818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:49:23.344464 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.253.194:48833 (local address 127.11.253.254:35823)
0806 03:49:23.344519 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0806 03:49:23.344522 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 03:49:23.344824 (+ 302us) server_negotiation.cc:407] Connection header received
0806 03:49:23.345523 (+ 699us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:49:23.345525 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:49:23.345539 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:49:23.345614 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:49:23.346516 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:49:23.347324 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:49:23.349014 (+ 1690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:49:23.349355 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:49:23.349436 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:49:23.352332 (+ 2896us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:49:23.352334 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:49:23.352335 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:49:23.352345 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:49:23.354354 (+ 2009us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:49:23.354835 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:49:23.354837 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:49:23.354837 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:49:23.354911 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:49:23.355193 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:49:23.355194 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:49:23.355194 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:49:23.355321 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:49:23.355401 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0806 03:49:23.355747 (+ 346us) server_negotiation.cc:299] Negotiation successful
0806 03:49:23.355847 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260806 03:49:23.356581 31013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:49:23.344447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:35823 (local address 127.11.253.194:48833)
0806 03:49:23.344643 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0806 03:49:23.344695 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0806 03:49:23.345389 (+ 694us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:49:23.345689 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:49:23.345694 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:49:23.346006 (+ 312us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:49:23.346307 (+ 301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:49:23.346314 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:49:23.347446 (+ 1132us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:49:23.347447 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:49:23.348906 (+ 1459us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:49:23.348910 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:49:23.349481 (+ 571us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:49:23.349483 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:49:23.349671 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:49:23.350294 (+ 623us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:49:23.350307 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:49:23.352178 (+ 1871us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:49:23.354447 (+ 2269us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:49:23.354453 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:49:23.354456 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:49:23.354732 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:49:23.354992 (+ 260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:49:23.354993 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:49:23.354993 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:49:23.355114 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:49:23.355391 (+ 277us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:49:23.355399 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:49:23.355572 (+ 173us) client_negotiation.cc:769] Sending connection context
0806 03:49:23.356107 (+ 535us) client_negotiation.cc:240] Negotiation successful
0806 03:49:23.356290 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":102,"threads_started":1}
I20260806 03:49:23.357285 31011 heartbeater.cc:347] Connected to a master server at 127.11.253.254:35823
I20260806 03:49:23.357402 31011 heartbeater.cc:464] Registering TS with master...
I20260806 03:49:23.357612 31011 heartbeater.cc:511] Master 127.11.253.254:35823 requested a full tablet report, sending...
W20260806 03:49:23.358410 31011 heartbeater.cc:644] Failed to heartbeat to 127.11.253.254:35823 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260806 03:49:23.358489 31011 heartbeater.cc:464] Registering TS with master...
I20260806 03:49:23.358559 31011 heartbeater.cc:511] Master 127.11.253.254:35823 requested a full tablet report, sending...
I20260806 03:49:23.358928 31011 heartbeater.cc:464] Registering TS with master...
I20260806 03:49:23.359004 31011 heartbeater.cc:511] Master 127.11.253.254:35823 requested a full tablet report, sending...
W20260806 03:49:23.359325 31011 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 03:49:23.487282 31014 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:49:23.487648 31014 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:49:23.487681 31014 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:49:23.489171 31014 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:49:23.489248 31014 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:49:23.489297 31014 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.253.195
I20260806 03:49:23.491462 31014 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.253.212:38447
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988004637995-12279-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988004637995-12279-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.11.253.212:34205/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.195.keytab
--principal=kudu/127.11.253.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.253.195:45445
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988004637995-12279-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.253.195
--webserver_port=34965
--tserver_master_addrs=127.11.253.254:35823
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31014
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.253.195
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988004637995-12279-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:49:23.491758 31014 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:49:23.492026 31014 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:49:23.498375 31024 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:49:23.498378 31020 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:49:23.498649 31019 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:49:23.499178 31014 server_base.cc:1034] running on GCE node
I20260806 03:49:23.499441 31014 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:49:23.499739 31014 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:49:23.500929 31014 hybrid_clock.cc:648] HybridClock initialized: now 1785988163500904 us; error 37 us; skew 500 ppm
Aug 06 03:49:23 dist-test-slave-6ntk krb5kdc[29119](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988163, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.253.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:49:23.504578 31014 init.cc:377] Logged in from keytab as kudu/127.11.253.195@KRBTEST.COM (short username kudu)
I20260806 03:49:23.505154 31014 webserver.cc:460] Webserver started at http://127.11.253.195:34965/ using document root <none> and password file <none>
I20260806 03:49:23.505479 31014 fs_manager.cc:359] Metadata directory not provided
I20260806 03:49:23.505566 31014 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 06 03:49:23 dist-test-slave-6ntk krb5kdc[29119](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988163, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.253.195@KRBTEST.COM for HTTP/127.11.253.212@KRBTEST.COM
I20260806 03:49:23.524422 31014 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.001s sys 0.000s
I20260806 03:49:23.525599 31035 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:49:23.525841 31014 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260806 03:49:23.526000 31014 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988004637995-12279-0/minicluster-data/ts-2/data,/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988004637995-12279-0/minicluster-data/ts-2/wal
uuid: "63b63eea39d4499fac09468b36792700"
format_stamp: "Formatted at 2026-08-06 03:49:22 on dist-test-slave-6ntk"
server_key: "c082b22a2cc035c21684e6a7a7bd692e"
server_key_iv: "b7f3e8d3b6fee7136ec050b01b4e1960"
server_key_version: "kuduclusterkey@0"
I20260806 03:49:23.526508 31014 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988004637995-12279-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988004637995-12279-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988004637995-12279-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:49:23.556593 31014 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:49:23.559430 31014 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:49:23.559687 31014 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:49:23.560030 31014 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 03:49:23.560484 31014 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 03:49:23.560566 31014 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 03:49:23.560627 31014 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 03:49:23.560674 31014 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 03:49:23.567760 31014 rpc_server.cc:307] RPC server started. Bound to: 127.11.253.195:45445
I20260806 03:49:23.568019 31186 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.253.195:45445 every 8 connection(s)
I20260806 03:49:23.568394 31014 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988004637995-12279-0/minicluster-data/ts-2/data/info.pb
I20260806 03:49:23.570183 12279 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskToVWRe/build/release/bin/kudu as pid 31014
Aug 06 03:49:23 dist-test-slave-6ntk krb5kdc[29119](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988163, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.253.195@KRBTEST.COM for kudu/127.11.253.254@KRBTEST.COM
I20260806 03:49:23.582293 30818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:49:23.569724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.253.195:42605 (local address 127.11.253.254:35823)
0806 03:49:23.569785 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0806 03:49:23.569787 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:49:23.570130 (+ 343us) server_negotiation.cc:407] Connection header received
0806 03:49:23.571468 (+ 1338us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:49:23.571469 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:49:23.571482 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:49:23.571550 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:49:23.572328 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:49:23.573073 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:49:23.574667 (+ 1594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:49:23.574943 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:49:23.574992 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:49:23.577901 (+ 2909us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:49:23.577903 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:49:23.577904 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:49:23.577912 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:49:23.580279 (+ 2367us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:49:23.581068 (+ 789us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:49:23.581069 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:49:23.581070 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:49:23.581133 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:49:23.581467 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:49:23.581468 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:49:23.581469 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:49:23.581611 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:49:23.581659 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0806 03:49:23.582094 (+ 435us) server_negotiation.cc:299] Negotiation successful
0806 03:49:23.582183 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260806 03:49:23.582914 31195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:49:23.570653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40382 (local address 127.11.253.254:35823)
0806 03:49:23.570807 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0806 03:49:23.570809 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:49:23.570871 (+ 62us) server_negotiation.cc:407] Connection header received
0806 03:49:23.570966 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:49:23.570967 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:49:23.570978 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:49:23.571075 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:49:23.571834 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:49:23.572361 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:49:23.577460 (+ 5099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:49:23.577789 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:49:23.577862 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:49:23.579422 (+ 1560us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:49:23.579424 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:49:23.579424 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:49:23.579431 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:49:23.581217 (+ 1786us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:49:23.581613 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:49:23.581614 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:49:23.581614 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:49:23.581672 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:49:23.581913 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:49:23.581914 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:49:23.581915 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:49:23.582090 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:49:23.582245 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0806 03:49:23.582718 (+ 473us) server_negotiation.cc:299] Negotiation successful
0806 03:49:23.582813 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":86,"threads_started":1}
I20260806 03:49:23.583485 31191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:49:23.569732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:35823 (local address 127.11.253.195:42605)
0806 03:49:23.569944 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0806 03:49:23.570001 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0806 03:49:23.571333 (+ 1332us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:49:23.571631 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:49:23.571636 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:49:23.571868 (+ 232us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:49:23.572186 (+ 318us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:49:23.572194 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:49:23.573197 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:49:23.573198 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:49:23.574536 (+ 1338us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:49:23.574540 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:49:23.575113 (+ 573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:49:23.575114 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:49:23.575289 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:49:23.575973 (+ 684us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:49:23.575984 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:49:23.577757 (+ 1773us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:49:23.580637 (+ 2880us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:49:23.580645 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:49:23.580647 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:49:23.580963 (+ 316us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:49:23.581214 (+ 251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:49:23.581215 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:49:23.581215 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:49:23.581379 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:49:23.581715 (+ 336us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:49:23.581722 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:49:23.581920 (+ 198us) client_negotiation.cc:769] Sending connection context
0806 03:49:23.582710 (+ 790us) client_negotiation.cc:240] Negotiation successful
0806 03:49:23.582898 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":83,"threads_started":1}
I20260806 03:49:23.584105 31188 heartbeater.cc:347] Connected to a master server at 127.11.253.254:35823
I20260806 03:49:23.584220 31188 heartbeater.cc:464] Registering TS with master...
I20260806 03:49:23.584419 31188 heartbeater.cc:511] Master 127.11.253.254:35823 requested a full tablet report, sending...
W20260806 03:49:23.585143 31188 heartbeater.cc:644] Failed to heartbeat to 127.11.253.254:35823 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260806 03:49:23.585227 31188 heartbeater.cc:464] Registering TS with master...
I20260806 03:49:23.585290 31188 heartbeater.cc:511] Master 127.11.253.254:35823 requested a full tablet report, sending...
I20260806 03:49:23.585664 31188 heartbeater.cc:464] Registering TS with master...
I20260806 03:49:23.585741 31188 heartbeater.cc:511] Master 127.11.253.254:35823 requested a full tablet report, sending...
W20260806 03:49:23.586136 31188 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 06 03:49:24 dist-test-slave-6ntk krb5kdc[29119](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1785988164, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.253.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:49:24.182600 30815 heartbeater.cc:464] Registering TS with master...
I20260806 03:49:24.182802 30815 heartbeater.cc:511] Master 127.11.253.254:35823 requested a full tablet report, sending...
I20260806 03:49:24.183312 30524 ts_manager.cc:194] Registered new tserver with Master: f19b134ef2dd4583a52f4e7168096c40 (127.11.253.193:45125)
I20260806 03:49:24.184386 30524 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.253.193@KRBTEST.COM'} at 127.11.253.193:44275
I20260806 03:49:24.359510 31011 heartbeater.cc:464] Registering TS with master...
I20260806 03:49:24.359683 31011 heartbeater.cc:511] Master 127.11.253.254:35823 requested a full tablet report, sending...
I20260806 03:49:24.363942 30524 ts_manager.cc:194] Registered new tserver with Master: 49f681b837fa45909ee3a260437772a6 (127.11.253.194:34161)
I20260806 03:49:24.364490 30524 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.253.194@KRBTEST.COM'} at 127.11.253.194:48833
Aug 06 03:49:24 dist-test-slave-6ntk krb5kdc[29119](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988164, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.253.254@KRBTEST.COM for HTTP/127.11.253.212@KRBTEST.COM
I20260806 03:49:24.586321 31188 heartbeater.cc:464] Registering TS with master...
I20260806 03:49:24.586498 31188 heartbeater.cc:511] Master 127.11.253.254:35823 requested a full tablet report, sending...
I20260806 03:49:24.586812 30524 ts_manager.cc:194] Registered new tserver with Master: 63b63eea39d4499fac09468b36792700 (127.11.253.195:45445)
I20260806 03:49:24.587407 30524 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.253.195@KRBTEST.COM'} at 127.11.253.195:42605
I20260806 03:49:24.597981 12279 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 03:49:24.608740 31220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:49:24.599005 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40388 (local address 127.11.253.254:35823)
0806 03:49:24.601974 (+ 2969us) server_negotiation.cc:206] Beginning negotiation
0806 03:49:24.601976 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:49:24.601985 (+ 9us) server_negotiation.cc:407] Connection header received
0806 03:49:24.602008 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:49:24.602009 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:49:24.602021 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:49:24.602098 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:49:24.602828 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:49:24.603375 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:49:24.604224 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:49:24.604435 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:49:24.604533 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:49:24.605811 (+ 1278us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:49:24.605813 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:49:24.605813 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:49:24.605820 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:49:24.607475 (+ 1655us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:49:24.607853 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:49:24.607855 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:49:24.607855 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:49:24.607917 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:49:24.608126 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:49:24.608127 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:49:24.608127 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:49:24.608243 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:49:24.608354 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0806 03:49:24.608461 (+ 107us) server_negotiation.cc:299] Negotiation successful
0806 03:49:24.608545 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2945,"thread_start_us":81,"threads_started":1}
I20260806 03:49:24.610864 30524 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40388:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 03:49:24.612617 30524 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 03:49:24.620961 31235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:49:24.615881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49234 (local address 127.11.253.195:45445)
0806 03:49:24.616025 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0806 03:49:24.616030 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 03:49:24.616476 (+ 446us) server_negotiation.cc:407] Connection header received
0806 03:49:24.616629 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:49:24.616633 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:49:24.616651 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:49:24.616731 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 03:49:24.617019 (+ 288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:49:24.618134 (+ 1115us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:49:24.619671 (+ 1537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:49:24.620443 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:49:24.620498 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:49:24.620559 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0806 03:49:24.620826 (+ 267us) server_negotiation.cc:299] Negotiation successful
0806 03:49:24.620878 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":91,"threads_started":1}
I20260806 03:49:24.621423 31232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:49:24.615421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.195:45445 (local address 127.0.0.1:49234)
0806 03:49:24.616406 (+ 985us) negotiation.cc:107] Waiting for socket to connect
0806 03:49:24.616417 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 03:49:24.616507 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:49:24.616831 (+ 324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:49:24.616835 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:49:24.616838 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 03:49:24.616926 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:49:24.616929 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:49:24.618240 (+ 1311us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:49:24.618241 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:49:24.619571 (+ 1330us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:49:24.619573 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:49:24.620620 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:49:24.620621 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:49:24.620708 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:49:24.620722 (+ 14us) client_negotiation.cc:769] Sending connection context
0806 03:49:24.621272 (+ 550us) client_negotiation.cc:240] Negotiation successful
0806 03:49:24.621331 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":967,"mutex_wait_us":24,"thread_start_us":88,"threads_started":1}
I20260806 03:49:24.622568 31100 tablet_service.cc:1467] Processing CreateTablet for tablet bddf21de9979405981ec01a0d1d6185b (DEFAULT_TABLE table=test-table [id=dcbdbed03f7e4325a333d76792db7eaf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 03:49:24.623001 31100 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bddf21de9979405981ec01a0d1d6185b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:49:24.623348 31229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:49:24.615414 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.193:45125 (local address 127.0.0.1:38096)
0806 03:49:24.615650 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0806 03:49:24.615693 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 03:49:24.615756 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:49:24.616836 (+ 1080us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:49:24.616836 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:49:24.616838 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 03:49:24.616926 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:49:24.616929 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:49:24.620893 (+ 3964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:49:24.620894 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:49:24.622132 (+ 1238us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:49:24.622135 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:49:24.623124 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:49:24.623125 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:49:24.623188 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:49:24.623192 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 03:49:24.623222 (+ 30us) client_negotiation.cc:240] Negotiation successful
0806 03:49:24.623258 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":208,"mutex_wait_us":3,"thread_start_us":94,"threads_started":1}
I20260806 03:49:24.624018 31237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:49:24.616167 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38096 (local address 127.11.253.193:45125)
0806 03:49:24.616672 (+ 505us) server_negotiation.cc:206] Beginning negotiation
0806 03:49:24.616680 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 03:49:24.616689 (+ 9us) server_negotiation.cc:407] Connection header received
0806 03:49:24.616723 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:49:24.616725 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:49:24.616739 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:49:24.617009 (+ 270us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 03:49:24.617085 (+ 76us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:49:24.618136 (+ 1051us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:49:24.622234 (+ 4098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:49:24.622997 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:49:24.623786 (+ 789us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:49:24.623872 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0806 03:49:24.623897 (+ 25us) server_negotiation.cc:299] Negotiation successful
0806 03:49:24.623943 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":472,"thread_start_us":51,"threads_started":1}
I20260806 03:49:24.624706 30720 tablet_service.cc:1467] Processing CreateTablet for tablet bddf21de9979405981ec01a0d1d6185b (DEFAULT_TABLE table=test-table [id=dcbdbed03f7e4325a333d76792db7eaf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 03:49:24.625066 30720 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bddf21de9979405981ec01a0d1d6185b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:49:24.629624 31240 tablet_bootstrap.cc:492] T bddf21de9979405981ec01a0d1d6185b P f19b134ef2dd4583a52f4e7168096c40: Bootstrap starting.
I20260806 03:49:24.629999 31231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:49:24.615648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.194:34161 (local address 127.0.0.1:51630)
0806 03:49:24.618525 (+ 2877us) negotiation.cc:107] Waiting for socket to connect
0806 03:49:24.618535 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 03:49:24.618587 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:49:24.618892 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:49:24.618893 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:49:24.618895 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 03:49:24.618931 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:49:24.618932 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:49:24.620132 (+ 1200us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:49:24.620133 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:49:24.628544 (+ 8411us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:49:24.628548 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:49:24.629656 (+ 1108us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:49:24.629657 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:49:24.629723 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:49:24.629729 (+ 6us) client_negotiation.cc:769] Sending connection context
0806 03:49:24.629821 (+ 92us) client_negotiation.cc:240] Negotiation successful
0806 03:49:24.629926 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2860,"thread_start_us":71,"threads_started":1}
I20260806 03:49:24.630467 31240 tablet_bootstrap.cc:654] T bddf21de9979405981ec01a0d1d6185b P f19b134ef2dd4583a52f4e7168096c40: Neither blocks nor log segments found. Creating new log.
I20260806 03:49:24.630544 31236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:49:24.616138 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51630 (local address 127.11.253.194:34161)
0806 03:49:24.616284 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0806 03:49:24.616288 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 03:49:24.618651 (+ 2363us) server_negotiation.cc:407] Connection header received
0806 03:49:24.618695 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:49:24.618697 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:49:24.618714 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:49:24.618783 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 03:49:24.619003 (+ 220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:49:24.620042 (+ 1039us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:49:24.628661 (+ 8619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:49:24.629483 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:49:24.629563 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:49:24.629618 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0806 03:49:24.630395 (+ 777us) server_negotiation.cc:299] Negotiation successful
0806 03:49:24.630449 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":59,"threads_started":1}
I20260806 03:49:24.630884 31240 log.cc:824] T bddf21de9979405981ec01a0d1d6185b P f19b134ef2dd4583a52f4e7168096c40: Log is configured to *not* fsync() on all Append() calls
I20260806 03:49:24.631129 31239 tablet_bootstrap.cc:492] T bddf21de9979405981ec01a0d1d6185b P 63b63eea39d4499fac09468b36792700: Bootstrap starting.
I20260806 03:49:24.631419 30921 tablet_service.cc:1467] Processing CreateTablet for tablet bddf21de9979405981ec01a0d1d6185b (DEFAULT_TABLE table=test-table [id=dcbdbed03f7e4325a333d76792db7eaf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 03:49:24.631796 31240 tablet_bootstrap.cc:492] T bddf21de9979405981ec01a0d1d6185b P f19b134ef2dd4583a52f4e7168096c40: No bootstrap required, opened a new log
I20260806 03:49:24.631800 30921 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bddf21de9979405981ec01a0d1d6185b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:49:24.631841 31239 tablet_bootstrap.cc:654] T bddf21de9979405981ec01a0d1d6185b P 63b63eea39d4499fac09468b36792700: Neither blocks nor log segments found. Creating new log.
I20260806 03:49:24.631863 31240 ts_tablet_manager.cc:1397] T bddf21de9979405981ec01a0d1d6185b P f19b134ef2dd4583a52f4e7168096c40: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 03:49:24.632160 31239 log.cc:824] T bddf21de9979405981ec01a0d1d6185b P 63b63eea39d4499fac09468b36792700: Log is configured to *not* fsync() on all Append() calls
I20260806 03:49:24.633553 31240 raft_consensus.cc:348] T bddf21de9979405981ec01a0d1d6185b P f19b134ef2dd4583a52f4e7168096c40 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f19b134ef2dd4583a52f4e7168096c40" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 45125 } } peers { permanent_uuid: "63b63eea39d4499fac09468b36792700" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 45445 } } peers { permanent_uuid: "49f681b837fa45909ee3a260437772a6" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 34161 } }
I20260806 03:49:24.633665 31240 raft_consensus.cc:374] T bddf21de9979405981ec01a0d1d6185b P f19b134ef2dd4583a52f4e7168096c40 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:49:24.633687 31240 raft_consensus.cc:729] T bddf21de9979405981ec01a0d1d6185b P f19b134ef2dd4583a52f4e7168096c40 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f19b134ef2dd4583a52f4e7168096c40, State: Initialized, Role: FOLLOWER
I20260806 03:49:24.633823 31240 consensus_queue.cc:260] T bddf21de9979405981ec01a0d1d6185b P f19b134ef2dd4583a52f4e7168096c40 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f19b134ef2dd4583a52f4e7168096c40" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 45125 } } peers { permanent_uuid: "63b63eea39d4499fac09468b36792700" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 45445 } } peers { permanent_uuid: "49f681b837fa45909ee3a260437772a6" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 34161 } }
I20260806 03:49:24.634095 31240 ts_tablet_manager.cc:1428] T bddf21de9979405981ec01a0d1d6185b P f19b134ef2dd4583a52f4e7168096c40: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 03:49:24.634296 30815 heartbeater.cc:503] Master 127.11.253.254:35823 was elected leader, sending a full tablet report...
I20260806 03:49:24.634538 31239 tablet_bootstrap.cc:492] T bddf21de9979405981ec01a0d1d6185b P 63b63eea39d4499fac09468b36792700: No bootstrap required, opened a new log
I20260806 03:49:24.634600 31239 ts_tablet_manager.cc:1397] T bddf21de9979405981ec01a0d1d6185b P 63b63eea39d4499fac09468b36792700: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260806 03:49:24.635469 31244 tablet_bootstrap.cc:492] T bddf21de9979405981ec01a0d1d6185b P 49f681b837fa45909ee3a260437772a6: Bootstrap starting.
I20260806 03:49:24.636132 31244 tablet_bootstrap.cc:654] T bddf21de9979405981ec01a0d1d6185b P 49f681b837fa45909ee3a260437772a6: Neither blocks nor log segments found. Creating new log.
I20260806 03:49:24.636489 31244 log.cc:824] T bddf21de9979405981ec01a0d1d6185b P 49f681b837fa45909ee3a260437772a6: Log is configured to *not* fsync() on all Append() calls
I20260806 03:49:24.637456 31239 raft_consensus.cc:348] T bddf21de9979405981ec01a0d1d6185b P 63b63eea39d4499fac09468b36792700 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f19b134ef2dd4583a52f4e7168096c40" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 45125 } } peers { permanent_uuid: "63b63eea39d4499fac09468b36792700" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 45445 } } peers { permanent_uuid: "49f681b837fa45909ee3a260437772a6" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 34161 } }
I20260806 03:49:24.637794 31244 tablet_bootstrap.cc:492] T bddf21de9979405981ec01a0d1d6185b P 49f681b837fa45909ee3a260437772a6: No bootstrap required, opened a new log
I20260806 03:49:24.637849 31239 raft_consensus.cc:374] T bddf21de9979405981ec01a0d1d6185b P 63b63eea39d4499fac09468b36792700 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:49:24.637921 31244 ts_tablet_manager.cc:1397] T bddf21de9979405981ec01a0d1d6185b P 49f681b837fa45909ee3a260437772a6: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 03:49:24.638003 31239 raft_consensus.cc:729] T bddf21de9979405981ec01a0d1d6185b P 63b63eea39d4499fac09468b36792700 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 63b63eea39d4499fac09468b36792700, State: Initialized, Role: FOLLOWER
I20260806 03:49:24.638230 31239 consensus_queue.cc:260] T bddf21de9979405981ec01a0d1d6185b P 63b63eea39d4499fac09468b36792700 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f19b134ef2dd4583a52f4e7168096c40" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 45125 } } peers { permanent_uuid: "63b63eea39d4499fac09468b36792700" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 45445 } } peers { permanent_uuid: "49f681b837fa45909ee3a260437772a6" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 34161 } }
I20260806 03:49:24.638617 31239 ts_tablet_manager.cc:1428] T bddf21de9979405981ec01a0d1d6185b P 63b63eea39d4499fac09468b36792700: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260806 03:49:24.638780 31188 heartbeater.cc:503] Master 127.11.253.254:35823 was elected leader, sending a full tablet report...
I20260806 03:49:24.640558 31244 raft_consensus.cc:348] T bddf21de9979405981ec01a0d1d6185b P 49f681b837fa45909ee3a260437772a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f19b134ef2dd4583a52f4e7168096c40" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 45125 } } peers { permanent_uuid: "63b63eea39d4499fac09468b36792700" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 45445 } } peers { permanent_uuid: "49f681b837fa45909ee3a260437772a6" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 34161 } }
I20260806 03:49:24.640709 31244 raft_consensus.cc:374] T bddf21de9979405981ec01a0d1d6185b P 49f681b837fa45909ee3a260437772a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:49:24.640779 31244 raft_consensus.cc:729] T bddf21de9979405981ec01a0d1d6185b P 49f681b837fa45909ee3a260437772a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49f681b837fa45909ee3a260437772a6, State: Initialized, Role: FOLLOWER
I20260806 03:49:24.640923 31244 consensus_queue.cc:260] T bddf21de9979405981ec01a0d1d6185b P 49f681b837fa45909ee3a260437772a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f19b134ef2dd4583a52f4e7168096c40" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 45125 } } peers { permanent_uuid: "63b63eea39d4499fac09468b36792700" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 45445 } } peers { permanent_uuid: "49f681b837fa45909ee3a260437772a6" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 34161 } }
I20260806 03:49:24.641227 31244 ts_tablet_manager.cc:1428] T bddf21de9979405981ec01a0d1d6185b P 49f681b837fa45909ee3a260437772a6: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 03:49:24.641251 31011 heartbeater.cc:503] Master 127.11.253.254:35823 was elected leader, sending a full tablet report...
I20260806 03:49:24.654578 31243 raft_consensus.cc:482] T bddf21de9979405981ec01a0d1d6185b P f19b134ef2dd4583a52f4e7168096c40 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 03:49:24.654845 31243 raft_consensus.cc:504] T bddf21de9979405981ec01a0d1d6185b P f19b134ef2dd4583a52f4e7168096c40 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f19b134ef2dd4583a52f4e7168096c40" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 45125 } } peers { permanent_uuid: "63b63eea39d4499fac09468b36792700" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 45445 } } peers { permanent_uuid: "49f681b837fa45909ee3a260437772a6" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 34161 } }
I20260806 03:49:24.655714 31243 leader_election.cc:288] T bddf21de9979405981ec01a0d1d6185b P f19b134ef2dd4583a52f4e7168096c40 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 63b63eea39d4499fac09468b36792700 (127.11.253.195:45445), 49f681b837fa45909ee3a260437772a6 (127.11.253.194:34161)
I20260806 03:49:24.659669 31235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:49:24.655680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.253.193:36973 (local address 127.11.253.195:45445)
0806 03:49:24.655727 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0806 03:49:24.655729 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:49:24.655945 (+ 216us) server_negotiation.cc:407] Connection header received
0806 03:49:24.656045 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:49:24.656046 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:49:24.656056 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:49:24.656088 (+ 32us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 03:49:24.656382 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:49:24.657120 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:49:24.658669 (+ 1549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:49:24.659238 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:49:24.659291 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:49:24.659301 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0806 03:49:24.659548 (+ 247us) server_negotiation.cc:299] Negotiation successful
0806 03:49:24.659593 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260806 03:49:24.659669 31236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:49:24.655680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.253.193:46573 (local address 127.11.253.194:34161)
0806 03:49:24.655763 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0806 03:49:24.655765 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:49:24.655928 (+ 163us) server_negotiation.cc:407] Connection header received
0806 03:49:24.656045 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:49:24.656046 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:49:24.656056 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:49:24.656088 (+ 32us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 03:49:24.656369 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:49:24.657127 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:49:24.658668 (+ 1541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:49:24.659248 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:49:24.659293 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:49:24.659301 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0806 03:49:24.659548 (+ 247us) server_negotiation.cc:299] Negotiation successful
0806 03:49:24.659593 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260806 03:49:24.659979 31254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:49:24.655430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.194:34161 (local address 127.11.253.193:46573)
0806 03:49:24.655840 (+ 410us) negotiation.cc:107] Waiting for socket to connect
0806 03:49:24.655868 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0806 03:49:24.655959 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:49:24.656161 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:49:24.656161 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:49:24.656164 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 03:49:24.656284 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:49:24.656286 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:49:24.657231 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:49:24.657232 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:49:24.658553 (+ 1321us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:49:24.658556 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:49:24.659382 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:49:24.659383 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:49:24.659458 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:49:24.659461 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 03:49:24.659816 (+ 355us) client_negotiation.cc:240] Negotiation successful
0806 03:49:24.659891 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":392,"mutex_wait_us":18,"thread_start_us":125,"threads_started":1}
I20260806 03:49:24.660262 31255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:49:24.655430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.195:45445 (local address 127.11.253.193:36973)
0806 03:49:24.655890 (+ 460us) negotiation.cc:107] Waiting for socket to connect
0806 03:49:24.655899 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 03:49:24.655967 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:49:24.656161 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:49:24.656161 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:49:24.656164 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 03:49:24.656284 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:49:24.656286 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:49:24.657231 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:49:24.657232 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:49:24.658553 (+ 1321us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:49:24.658556 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:49:24.659389 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:49:24.659390 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:49:24.659458 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:49:24.659463 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 03:49:24.660121 (+ 658us) client_negotiation.cc:240] Negotiation successful
0806 03:49:24.660163 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":444,"mutex_wait_us":125,"thread_start_us":115,"threads_started":1}
I20260806 03:49:24.660451 30946 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bddf21de9979405981ec01a0d1d6185b" candidate_uuid: "f19b134ef2dd4583a52f4e7168096c40" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49f681b837fa45909ee3a260437772a6" is_pre_election: true
I20260806 03:49:24.660634 30946 raft_consensus.cc:2454] T bddf21de9979405981ec01a0d1d6185b P 49f681b837fa45909ee3a260437772a6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f19b134ef2dd4583a52f4e7168096c40 in term 0.
I20260806 03:49:24.660590 31128 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bddf21de9979405981ec01a0d1d6185b" candidate_uuid: "f19b134ef2dd4583a52f4e7168096c40" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "63b63eea39d4499fac09468b36792700" is_pre_election: true
I20260806 03:49:24.660784 31128 raft_consensus.cc:2454] T bddf21de9979405981ec01a0d1d6185b P 63b63eea39d4499fac09468b36792700 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f19b134ef2dd4583a52f4e7168096c40 in term 0.
I20260806 03:49:24.660904 30646 leader_election.cc:302] T bddf21de9979405981ec01a0d1d6185b P f19b134ef2dd4583a52f4e7168096c40 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49f681b837fa45909ee3a260437772a6, f19b134ef2dd4583a52f4e7168096c40; no voters:
I20260806 03:49:24.661044 31243 raft_consensus.cc:2781] T bddf21de9979405981ec01a0d1d6185b P f19b134ef2dd4583a52f4e7168096c40 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 03:49:24.661108 31243 raft_consensus.cc:482] T bddf21de9979405981ec01a0d1d6185b P f19b134ef2dd4583a52f4e7168096c40 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 03:49:24.661136 31243 raft_consensus.cc:3037] T bddf21de9979405981ec01a0d1d6185b P f19b134ef2dd4583a52f4e7168096c40 [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:49:24.662034 31243 raft_consensus.cc:504] T bddf21de9979405981ec01a0d1d6185b P f19b134ef2dd4583a52f4e7168096c40 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f19b134ef2dd4583a52f4e7168096c40" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 45125 } } peers { permanent_uuid: "63b63eea39d4499fac09468b36792700" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 45445 } } peers { permanent_uuid: "49f681b837fa45909ee3a260437772a6" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 34161 } }
I20260806 03:49:24.662212 31243 leader_election.cc:288] T bddf21de9979405981ec01a0d1d6185b P f19b134ef2dd4583a52f4e7168096c40 [CANDIDATE]: Term 1 election: Requested vote from peers 63b63eea39d4499fac09468b36792700 (127.11.253.195:45445), 49f681b837fa45909ee3a260437772a6 (127.11.253.194:34161)
I20260806 03:49:24.662408 31128 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bddf21de9979405981ec01a0d1d6185b" candidate_uuid: "f19b134ef2dd4583a52f4e7168096c40" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "63b63eea39d4499fac09468b36792700"
I20260806 03:49:24.662461 30946 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bddf21de9979405981ec01a0d1d6185b" candidate_uuid: "f19b134ef2dd4583a52f4e7168096c40" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49f681b837fa45909ee3a260437772a6"
I20260806 03:49:24.662494 31128 raft_consensus.cc:3037] T bddf21de9979405981ec01a0d1d6185b P 63b63eea39d4499fac09468b36792700 [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:49:24.662534 30946 raft_consensus.cc:3037] T bddf21de9979405981ec01a0d1d6185b P 49f681b837fa45909ee3a260437772a6 [term 0 FOLLOWER]: Advancing to term 1
W20260806 03:49:24.663141 30816 tablet.cc:2369] T bddf21de9979405981ec01a0d1d6185b P f19b134ef2dd4583a52f4e7168096c40: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 03:49:24.663410 30946 raft_consensus.cc:2454] T bddf21de9979405981ec01a0d1d6185b P 49f681b837fa45909ee3a260437772a6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f19b134ef2dd4583a52f4e7168096c40 in term 1.
I20260806 03:49:24.663410 31128 raft_consensus.cc:2454] T bddf21de9979405981ec01a0d1d6185b P 63b63eea39d4499fac09468b36792700 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f19b134ef2dd4583a52f4e7168096c40 in term 1.
I20260806 03:49:24.663604 30648 leader_election.cc:302] T bddf21de9979405981ec01a0d1d6185b P f19b134ef2dd4583a52f4e7168096c40 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 63b63eea39d4499fac09468b36792700, f19b134ef2dd4583a52f4e7168096c40; no voters:
I20260806 03:49:24.663808 31243 raft_consensus.cc:2781] T bddf21de9979405981ec01a0d1d6185b P f19b134ef2dd4583a52f4e7168096c40 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 03:49:24.664024 31243 raft_consensus.cc:686] T bddf21de9979405981ec01a0d1d6185b P f19b134ef2dd4583a52f4e7168096c40 [term 1 LEADER]: Becoming Leader. State: Replica: f19b134ef2dd4583a52f4e7168096c40, State: Running, Role: LEADER
I20260806 03:49:24.664124 31243 consensus_queue.cc:237] T bddf21de9979405981ec01a0d1d6185b P f19b134ef2dd4583a52f4e7168096c40 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f19b134ef2dd4583a52f4e7168096c40" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 45125 } } peers { permanent_uuid: "63b63eea39d4499fac09468b36792700" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 45445 } } peers { permanent_uuid: "49f681b837fa45909ee3a260437772a6" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 34161 } }
I20260806 03:49:24.664935 30524 catalog_manager.cc:5495] T bddf21de9979405981ec01a0d1d6185b P f19b134ef2dd4583a52f4e7168096c40 reported cstate change: term changed from 0 to 1, leader changed from <none> to f19b134ef2dd4583a52f4e7168096c40 (127.11.253.193). New cstate: current_term: 1 leader_uuid: "f19b134ef2dd4583a52f4e7168096c40" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f19b134ef2dd4583a52f4e7168096c40" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 45125 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "63b63eea39d4499fac09468b36792700" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 45445 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49f681b837fa45909ee3a260437772a6" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 34161 } health_report { overall_health: UNKNOWN } } }
I20260806 03:49:24.680934 31237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:49:24.678478 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38106 (local address 127.11.253.193:45125)
0806 03:49:24.678522 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0806 03:49:24.678524 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:49:24.678567 (+ 43us) server_negotiation.cc:407] Connection header received
0806 03:49:24.678645 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:49:24.678645 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:49:24.678664 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:49:24.678751 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 03:49:24.678889 (+ 138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:49:24.679354 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:49:24.680132 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:49:24.680342 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:49:24.680425 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:49:24.680595 (+ 170us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 03:49:24.680706 (+ 111us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 03:49:24.680803 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0806 03:49:24.680829 (+ 26us) server_negotiation.cc:299] Negotiation successful
0806 03:49:24.680849 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260806 03:49:24.683153 31128 raft_consensus.cc:1261] T bddf21de9979405981ec01a0d1d6185b P 63b63eea39d4499fac09468b36792700 [term 1 FOLLOWER]: Refusing update from remote peer f19b134ef2dd4583a52f4e7168096c40: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 03:49:24.683171 30946 raft_consensus.cc:1261] T bddf21de9979405981ec01a0d1d6185b P 49f681b837fa45909ee3a260437772a6 [term 1 FOLLOWER]: Refusing update from remote peer f19b134ef2dd4583a52f4e7168096c40: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 03:49:24.683610 31256 consensus_queue.cc:1035] T bddf21de9979405981ec01a0d1d6185b P f19b134ef2dd4583a52f4e7168096c40 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49f681b837fa45909ee3a260437772a6" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 34161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 03:49:24.683735 31243 consensus_queue.cc:1035] T bddf21de9979405981ec01a0d1d6185b P f19b134ef2dd4583a52f4e7168096c40 [LEADER]: Connected to new peer: Peer: permanent_uuid: "63b63eea39d4499fac09468b36792700" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 45445 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 03:49:24.687299 31266 mvcc.cc:205] Tried to move back new op lower bound from 7315407522539409408 to 7315407522464903168. Current Snapshot: MvccSnapshot[applied={T|T < 7315407522539409408}]
I20260806 03:49:24.688831 31265 mvcc.cc:205] Tried to move back new op lower bound from 7315407522539409408 to 7315407522464903168. Current Snapshot: MvccSnapshot[applied={T|T < 7315407522539409408}]
I20260806 03:49:24.692368 30524 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40388:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 03:49:24.692469 30524 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40388:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 03:49:24.693174 30524 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f648b4f531da41c9a52f2a7b07c56fe3: Sending DeleteTablet for 3 replicas of tablet bddf21de9979405981ec01a0d1d6185b
I20260806 03:49:24.693672 30720 tablet_service.cc:1514] Processing DeleteTablet for tablet bddf21de9979405981ec01a0d1d6185b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 03:49:24 UTC) from {username='kudu', principal='kudu/127.11.253.254@KRBTEST.COM'} at 127.0.0.1:38096
I20260806 03:49:24.693760 12279 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskToVWRe/build/release/bin/kudu with pid 30581
I20260806 03:49:24.693918 30921 tablet_service.cc:1514] Processing DeleteTablet for tablet bddf21de9979405981ec01a0d1d6185b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 03:49:24 UTC) from {username='kudu', principal='kudu/127.11.253.254@KRBTEST.COM'} at 127.0.0.1:51630
I20260806 03:49:24.694235 31100 tablet_service.cc:1514] Processing DeleteTablet for tablet bddf21de9979405981ec01a0d1d6185b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 03:49:24 UTC) from {username='kudu', principal='kudu/127.11.253.254@KRBTEST.COM'} at 127.0.0.1:49234
I20260806 03:49:24.701525 31276 tablet_replica.cc:330] T bddf21de9979405981ec01a0d1d6185b P 63b63eea39d4499fac09468b36792700: stopping tablet replica
I20260806 03:49:24.701709 31276 raft_consensus.cc:2229] T bddf21de9979405981ec01a0d1d6185b P 63b63eea39d4499fac09468b36792700 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 03:49:24.701905 31276 raft_consensus.cc:2258] T bddf21de9979405981ec01a0d1d6185b P 63b63eea39d4499fac09468b36792700 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 03:49:24.702466 31276 ts_tablet_manager.cc:1905] T bddf21de9979405981ec01a0d1d6185b P 63b63eea39d4499fac09468b36792700: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 03:49:24.704787 31276 ts_tablet_manager.cc:1918] T bddf21de9979405981ec01a0d1d6185b P 63b63eea39d4499fac09468b36792700: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 03:49:24.704921 31276 log.cc:1196] T bddf21de9979405981ec01a0d1d6185b P 63b63eea39d4499fac09468b36792700: Deleting WAL directory at /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988004637995-12279-0/minicluster-data/ts-2/wal/wals/bddf21de9979405981ec01a0d1d6185b
I20260806 03:49:24.705268 31276 ts_tablet_manager.cc:1939] T bddf21de9979405981ec01a0d1d6185b P 63b63eea39d4499fac09468b36792700: Deleting consensus metadata
I20260806 03:49:24.705765 30505 catalog_manager.cc:4841] TS 63b63eea39d4499fac09468b36792700 (127.11.253.195:45445): tablet bddf21de9979405981ec01a0d1d6185b (table test-table [id=dcbdbed03f7e4325a333d76792db7eaf]) successfully deleted
W20260806 03:49:24.713564 30502 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.253.193:45125 (error 108)
I20260806 03:49:24.713758 12279 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskToVWRe/build/release/bin/kudu with pid 30821
I20260806 03:49:24.730265 31231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:49:24.730087 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.193:45125 (local address 127.0.0.1:38118)
0806 03:49:24.730164 (+ 77us) negotiation.cc:107] Waiting for socket to connect
0806 03:49:24.730217 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.253.193:45125: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260806 03:49:24.730501 30502 catalog_manager.cc:4584] TS f19b134ef2dd4583a52f4e7168096c40 (127.11.253.193:45125): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet bddf21de9979405981ec01a0d1d6185b: Network error: Client connection negotiation failed: client connection to 127.11.253.193:45125: connect: Connection refused (error 111)
I20260806 03:49:24.735872 12279 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskToVWRe/build/release/bin/kudu with pid 31014
W20260806 03:49:24.737743 31231 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 03:49:24.735436 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.194:34161 (local address 127.0.0.1:51634)
0806 03:49:24.737634 (+ 2198us) negotiation.cc:107] Waiting for socket to connect
0806 03:49:24.737674 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.253.194:34161: connect: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":2172}
I20260806 03:49:24.758157 12279 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskToVWRe/build/release/bin/kudu with pid 30478
I20260806 03:49:24.769390 12279 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260806 03:49:25.114420 12279 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260806 03:49:25.114542 12279 mini_ranger.cc:66] Stopping Ranger...
I20260806 03:49:25.459496 12279 mini_ranger.cc:68] Stopped Ranger
2026-08-06 03:49:25.459 UTC [29162] LOG: received smart shutdown request
2026-08-06 03:49:25.465 UTC [29162] LOG: background worker "logical replication launcher" (PID 29174) exited with exit code 1
2026-08-06 03:49:25.465 UTC [29166] LOG: shutting down
2026-08-06 03:49:25.648 UTC [29162] LOG: database system is shut down
2026-08-06T03:49:25Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (50614 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785988004637995-12279-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 03:49:25 dist-test-slave-6ntk krb5kdc[31291](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 03:49:25 dist-test-slave-6ntk krb5kdc[31291](info): set up 2 sockets
Aug 06 03:49:25 dist-test-slave-6ntk krb5kdc[31291](info): commencing operation
krb5kdc: starting...
W20260806 03:49:26.980070 12279 mini_kdc.cc:121] Time spent starting KDC: real 1.195s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785988004637995-12279-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785988004637995-12279-0/krb5kdc/test-user.keytab.
Aug 06 03:49:27 dist-test-slave-6ntk krb5kdc[31291](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988167, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T03:49:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T03:49:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.253.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.253.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.253.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.254.keytab.
Entry for principal kudu/127.11.253.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.254.keytab.
I20260806 03:49:27.112967 12279 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskToVWRe/build/release/bin/kudu
/tmp/dist-test-taskToVWRe/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785988004637995-12279-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785988004637995-12279-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785988004637995-12279-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785988004637995-12279-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.253.254:42495
--webserver_interface=127.11.253.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.254.keytab
--principal=kudu/127.11.253.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.253.212:40029
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.253.254:42495,127.11.253.253:35689,127.11.253.252:44101
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785988004637995-12279-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785988004637995-12279-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785988004637995-12279-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:49:27.214859 31305 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:49:27.215235 31305 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:49:27.215293 31305 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:49:27.217469 31305 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 03:49:27.217540 31305 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:49:27.217568 31305 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:49:27.217592 31305 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 03:49:27.217655 31305 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 03:49:27.220244 31305 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.253.212:40029
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785988004637995-12279-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785988004637995-12279-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.253.254:42495,127.11.253.253:35689,127.11.253.252:44101
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.254.keytab
--principal=kudu/127.11.253.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.253.254:42495
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785988004637995-12279-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.253.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31305
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785988004637995-12279-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:49:27.220533 31305 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:49:27.220903 31305 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:49:27.228260 31313 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:49:27.228307 31311 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:49:27.228278 31315 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:49:27.229007 31305 server_base.cc:1034] running on GCE node
I20260806 03:49:27.229283 31305 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:49:27.229578 31305 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:49:27.230741 31305 hybrid_clock.cc:648] HybridClock initialized: now 1785988167230722 us; error 42 us; skew 500 ppm
Aug 06 03:49:27 dist-test-slave-6ntk krb5kdc[31291](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988167, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.253.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:49:27.233299 31305 init.cc:377] Logged in from keytab as kudu/127.11.253.254@KRBTEST.COM (short username kudu)
I20260806 03:49:27.233691 31305 webserver.cc:460] Webserver started at http://127.11.253.254:33917/ using document root <none> and password file <none>
I20260806 03:49:27.233999 31305 fs_manager.cc:359] Metadata directory not provided
I20260806 03:49:27.234050 31305 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:49:27.234193 31305 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:49:27.235289 31305 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785988004637995-12279-0/minicluster-data/master-0/data/instance:
uuid: "8400f5fc4fda4d069125f49070b47383"
format_stamp: "Formatted at 2026-08-06 03:49:27 on dist-test-slave-6ntk"
server_key: "e8c8f71067d29a2133fc3a1554121277"
server_key_iv: "794886d7053e43804f4e05a84d376f5d"
server_key_version: "encryptionkey@0"
I20260806 03:49:27.235673 31305 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785988004637995-12279-0/minicluster-data/master-0/wal/instance:
uuid: "8400f5fc4fda4d069125f49070b47383"
format_stamp: "Formatted at 2026-08-06 03:49:27 on dist-test-slave-6ntk"
server_key: "e8c8f71067d29a2133fc3a1554121277"
server_key_iv: "794886d7053e43804f4e05a84d376f5d"
server_key_version: "encryptionkey@0"
I20260806 03:49:27.237105 31305 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 03:49:27.237923 31323 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:49:27.238119 31305 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 03:49:27.238222 31305 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785988004637995-12279-0/minicluster-data/master-0/data,/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785988004637995-12279-0/minicluster-data/master-0/wal
uuid: "8400f5fc4fda4d069125f49070b47383"
format_stamp: "Formatted at 2026-08-06 03:49:27 on dist-test-slave-6ntk"
server_key: "e8c8f71067d29a2133fc3a1554121277"
server_key_iv: "794886d7053e43804f4e05a84d376f5d"
server_key_version: "encryptionkey@0"
I20260806 03:49:27.238308 31305 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785988004637995-12279-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785988004637995-12279-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785988004637995-12279-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:49:27.255517 31305 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:49:27.257421 31305 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:49:27.257589 31305 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:49:27.261838 31305 rpc_server.cc:307] RPC server started. Bound to: 127.11.253.254:42495
I20260806 03:49:27.261936 31400 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.253.254:42495 every 8 connection(s)
I20260806 03:49:27.262327 31305 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785988004637995-12279-0/minicluster-data/master-0/data/info.pb
I20260806 03:49:27.263350 31403 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:49:27.265048 31403 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.253.254" port: 42495 } has no permanent_uuid. Determining permanent_uuid...
I20260806 03:49:27.269790 12279 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskToVWRe/build/release/bin/kudu as pid 31305
I20260806 03:49:27.269963 12279 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785988004637995-12279-0/minicluster-data/master-0/wal/instance
I20260806 03:49:27.270191 12279 external_mini_cluster.cc:1428] Setting key c2e2dd3a4df8b00b19d6103f7e38385d
Aug 06 03:49:27 dist-test-slave-6ntk krb5kdc[31291](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988167, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.253.254@KRBTEST.COM for kudu/127.11.253.254@KRBTEST.COM
I20260806 03:49:27.275720 31406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:49:27.265841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39924 (local address 127.11.253.254:42495)
0806 03:49:27.266086 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0806 03:49:27.266095 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 03:49:27.266268 (+ 173us) server_negotiation.cc:407] Connection header received
0806 03:49:27.266850 (+ 582us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:49:27.266852 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:49:27.266871 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:49:27.266962 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:49:27.267542 (+ 580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:49:27.268365 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:49:27.269186 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:49:27.269529 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:49:27.269611 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:49:27.272269 (+ 2658us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:49:27.272277 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:49:27.272277 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:49:27.272284 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:49:27.273850 (+ 1566us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:49:27.274439 (+ 589us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:49:27.274442 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:49:27.274442 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:49:27.274523 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:49:27.274793 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:49:27.274793 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:49:27.274794 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:49:27.274993 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:49:27.275054 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0806 03:49:27.275330 (+ 276us) server_negotiation.cc:299] Negotiation successful
0806 03:49:27.275532 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":106,"server-negotiator.queue_time_us":192,"thread_start_us":159,"threads_started":1}
I20260806 03:49:27.275733 31407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:49:27.265787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:42495 (local address 127.0.0.1:39924)
0806 03:49:27.266123 (+ 336us) negotiation.cc:107] Waiting for socket to connect
0806 03:49:27.266165 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 03:49:27.266666 (+ 501us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:49:27.266996 (+ 330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:49:27.266998 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:49:27.267260 (+ 262us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:49:27.267450 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:49:27.267454 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:49:27.268448 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:49:27.268449 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:49:27.269115 (+ 666us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:49:27.269117 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:49:27.269658 (+ 541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:49:27.269659 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:49:27.269747 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:49:27.270434 (+ 687us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:49:27.270444 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:49:27.272160 (+ 1716us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:49:27.273985 (+ 1825us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:49:27.273992 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:49:27.273994 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:49:27.274356 (+ 362us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:49:27.274601 (+ 245us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:49:27.274602 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:49:27.274602 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:49:27.274725 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:49:27.275063 (+ 338us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:49:27.275070 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:49:27.275146 (+ 76us) client_negotiation.cc:769] Sending connection context
0806 03:49:27.275260 (+ 114us) client_negotiation.cc:240] Negotiation successful
0806 03:49:27.275466 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":299,"thread_start_us":263,"threads_started":1}
I20260806 03:49:27.276881 31403 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.253.253" port: 35689 } has no permanent_uuid. Determining permanent_uuid...
I20260806 03:49:27.277299 31407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:49:27.277173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.253:35689 (local address 127.0.0.1:42810)
0806 03:49:27.277232 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0806 03:49:27.277279 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.253.253:35689: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260806 03:49:27.277465 31328 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.11.253.253:35689: connect: Connection refused (error 111)
I20260806 03:49:27.277983 31407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:49:27.277836 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.253:35689 (local address 127.0.0.1:42826)
0806 03:49:27.277920 (+ 84us) negotiation.cc:107] Waiting for socket to connect
0806 03:49:27.277952 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.253.253:35689: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
W20260806 03:49:27.278122 31403 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.253.253:35689: Network error: Client connection negotiation failed: client connection to 127.11.253.253:35689: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.11.253.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.253.253@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.253.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.253.keytab.
Entry for principal kudu/127.11.253.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.253.keytab.
I20260806 03:49:27.301182 12279 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskToVWRe/build/release/bin/kudu
/tmp/dist-test-taskToVWRe/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785988004637995-12279-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785988004637995-12279-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785988004637995-12279-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785988004637995-12279-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.253.253:35689
--webserver_interface=127.11.253.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.253.keytab
--principal=kudu/127.11.253.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.253.212:40029
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.253.254:42495,127.11.253.253:35689,127.11.253.252:44101
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785988004637995-12279-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785988004637995-12279-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785988004637995-12279-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 03:49:27.327435 31403 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.253.253" port: 35689 } attempt: 1
I20260806 03:49:27.327945 31407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:49:27.327803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.253:35689 (local address 127.0.0.1:42842)
0806 03:49:27.327869 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0806 03:49:27.327913 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.253.253:35689: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260806 03:49:27.328297 31407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:49:27.328198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.253:35689 (local address 127.0.0.1:42852)
0806 03:49:27.328246 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0806 03:49:27.328272 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.253.253:35689: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260806 03:49:27.328397 31403 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.253.253:35689: Network error: Client connection negotiation failed: client connection to 127.11.253.253:35689: connect: Connection refused (error 111)
W20260806 03:49:27.386253 31411 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:49:27.386500 31411 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:49:27.386538 31411 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:49:27.387645 31411 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 03:49:27.387715 31411 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:49:27.387742 31411 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:49:27.387763 31411 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 03:49:27.387796 31411 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 03:49:27.389256 31411 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.253.212:40029
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785988004637995-12279-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785988004637995-12279-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.11.253.254:42495,127.11.253.253:35689,127.11.253.252:44101
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.253.keytab
--principal=kudu/127.11.253.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.253.253:35689
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785988004637995-12279-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.11.253.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31411
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785988004637995-12279-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:49:27.389539 31411 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:49:27.389772 31411 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:49:27.394687 31423 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:49:27.394698 31417 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:49:27.394704 31418 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:49:27.395027 31411 server_base.cc:1034] running on GCE node
I20260806 03:49:27.395282 31411 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:49:27.395515 31411 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:49:27.396539 31403 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.253.253" port: 35689 } attempt: 2
I20260806 03:49:27.396659 31411 hybrid_clock.cc:648] HybridClock initialized: now 1785988167396636 us; error 38 us; skew 500 ppm
I20260806 03:49:27.396974 31407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:49:27.396811 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.253:35689 (local address 127.0.0.1:42860)
0806 03:49:27.396887 (+ 76us) negotiation.cc:107] Waiting for socket to connect
0806 03:49:27.396934 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.253.253:35689: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260806 03:49:27.397364 31407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:49:27.397259 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.253:35689 (local address 127.0.0.1:42874)
0806 03:49:27.397312 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0806 03:49:27.397339 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.253.253:35689: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260806 03:49:27.397733 31403 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.253.253:35689: Network error: Client connection negotiation failed: client connection to 127.11.253.253:35689: connect: Connection refused (error 111)
Aug 06 03:49:27 dist-test-slave-6ntk krb5kdc[31291](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988167, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.253.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:49:27.399428 31411 init.cc:377] Logged in from keytab as kudu/127.11.253.253@KRBTEST.COM (short username kudu)
I20260806 03:49:27.399885 31411 webserver.cc:460] Webserver started at http://127.11.253.253:34387/ using document root <none> and password file <none>
I20260806 03:49:27.400071 31411 fs_manager.cc:359] Metadata directory not provided
I20260806 03:49:27.400115 31411 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:49:27.400206 31411 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:49:27.401214 31411 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785988004637995-12279-0/minicluster-data/master-1/data/instance:
uuid: "e6f42db43a2b4a3dbf9bf81900be56c3"
format_stamp: "Formatted at 2026-08-06 03:49:27 on dist-test-slave-6ntk"
server_key: "a709ac3dc835adc2179fcc168fdcb024"
server_key_iv: "529e29febb39992fe3310ccaba787ad8"
server_key_version: "encryptionkey@0"
I20260806 03:49:27.401515 31411 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785988004637995-12279-0/minicluster-data/master-1/wal/instance:
uuid: "e6f42db43a2b4a3dbf9bf81900be56c3"
format_stamp: "Formatted at 2026-08-06 03:49:27 on dist-test-slave-6ntk"
server_key: "a709ac3dc835adc2179fcc168fdcb024"
server_key_iv: "529e29febb39992fe3310ccaba787ad8"
server_key_version: "encryptionkey@0"
I20260806 03:49:27.402958 31411 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260806 03:49:27.403700 31433 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:49:27.403900 31411 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260806 03:49:27.404012 31411 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785988004637995-12279-0/minicluster-data/master-1/data,/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785988004637995-12279-0/minicluster-data/master-1/wal
uuid: "e6f42db43a2b4a3dbf9bf81900be56c3"
format_stamp: "Formatted at 2026-08-06 03:49:27 on dist-test-slave-6ntk"
server_key: "a709ac3dc835adc2179fcc168fdcb024"
server_key_iv: "529e29febb39992fe3310ccaba787ad8"
server_key_version: "encryptionkey@0"
I20260806 03:49:27.404088 31411 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785988004637995-12279-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785988004637995-12279-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785988004637995-12279-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:49:27.418223 31411 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:49:27.420106 31411 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:49:27.420285 31411 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:49:27.424669 31411 rpc_server.cc:307] RPC server started. Bound to: 127.11.253.253:35689
I20260806 03:49:27.424727 31516 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.253.253:35689 every 8 connection(s)
I20260806 03:49:27.425107 31411 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785988004637995-12279-0/minicluster-data/master-1/data/info.pb
I20260806 03:49:27.426090 31517 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:49:27.426607 12279 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskToVWRe/build/release/bin/kudu as pid 31411
I20260806 03:49:27.426731 12279 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785988004637995-12279-0/minicluster-data/master-1/wal/instance
I20260806 03:49:27.426919 12279 external_mini_cluster.cc:1428] Setting key 8d238617e21f87e83db5e63ca5f69a0e
I20260806 03:49:27.428526 31517 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.253.254" port: 42495 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 03:49:27 dist-test-slave-6ntk krb5kdc[31291](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988167, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.253.253@KRBTEST.COM for kudu/127.11.253.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.11.253.252@KRBTEST.COM; defaulting to no policy
I20260806 03:49:27.438210 31406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:49:27.429456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39940 (local address 127.11.253.254:42495)
0806 03:49:27.429516 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0806 03:49:27.429518 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:49:27.429706 (+ 188us) server_negotiation.cc:407] Connection header received
0806 03:49:27.430275 (+ 569us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:49:27.430277 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:49:27.430291 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:49:27.430363 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:49:27.430996 (+ 633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:49:27.431569 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:49:27.432564 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:49:27.432782 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:49:27.432865 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:49:27.435386 (+ 2521us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:49:27.435389 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:49:27.435390 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:49:27.435404 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:49:27.436823 (+ 1419us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:49:27.437259 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:49:27.437260 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:49:27.437260 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:49:27.437311 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:49:27.437576 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:49:27.437577 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:49:27.437577 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:49:27.437679 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:49:27.437729 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0806 03:49:27.438078 (+ 349us) server_negotiation.cc:299] Negotiation successful
0806 03:49:27.438143 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.253.252@KRBTEST.COM" created.
I20260806 03:49:27.438550 31522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:49:27.429423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:42495 (local address 127.0.0.1:39940)
0806 03:49:27.429584 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0806 03:49:27.429620 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0806 03:49:27.430167 (+ 547us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:49:27.430438 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:49:27.430441 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:49:27.430689 (+ 248us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:49:27.430888 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:49:27.430892 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:49:27.431667 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:49:27.431668 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:49:27.432471 (+ 803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:49:27.432473 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:49:27.432876 (+ 403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:49:27.432877 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:49:27.433002 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:49:27.433519 (+ 517us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:49:27.433528 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:49:27.435273 (+ 1745us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:49:27.436922 (+ 1649us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:49:27.436929 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:49:27.436931 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:49:27.437174 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:49:27.437395 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:49:27.437397 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:49:27.437397 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:49:27.437509 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:49:27.437765 (+ 256us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:49:27.437772 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:49:27.437957 (+ 185us) client_negotiation.cc:769] Sending connection context
0806 03:49:27.438050 (+ 93us) client_negotiation.cc:240] Negotiation successful
0806 03:49:27.438287 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":78,"threads_started":1}
I20260806 03:49:27.439164 31517 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.253.253" port: 35689 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 03:49:27 dist-test-slave-6ntk krb5kdc[31291](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988167, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.253.253@KRBTEST.COM for kudu/127.11.253.253@KRBTEST.COM
I20260806 03:49:27.447382 31524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:49:27.439533 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42878 (local address 127.11.253.253:35689)
0806 03:49:27.439693 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0806 03:49:27.439697 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 03:49:27.439715 (+ 18us) server_negotiation.cc:407] Connection header received
0806 03:49:27.439734 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:49:27.439735 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:49:27.439755 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:49:27.439831 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:49:27.440160 (+ 329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:49:27.440883 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:49:27.441751 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:49:27.442190 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:49:27.442278 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:49:27.444417 (+ 2139us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:49:27.444425 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:49:27.444425 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:49:27.444431 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:49:27.446063 (+ 1632us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:49:27.446492 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:49:27.446495 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:49:27.446495 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:49:27.446550 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:49:27.446761 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:49:27.446761 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:49:27.446761 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:49:27.446906 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:49:27.446969 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0806 03:49:27.447138 (+ 169us) server_negotiation.cc:299] Negotiation successful
0806 03:49:27.447272 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":69,"server-negotiator.queue_time_us":132,"thread_start_us":107,"threads_started":1}
I20260806 03:49:27.447381 31522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:49:27.439456 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.253:35689 (local address 127.0.0.1:42878)
0806 03:49:27.439506 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0806 03:49:27.439518 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 03:49:27.439558 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:49:27.439849 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:49:27.439850 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:49:27.439993 (+ 143us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:49:27.440065 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:49:27.440067 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:49:27.440992 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:49:27.440993 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:49:27.441649 (+ 656us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:49:27.441651 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:49:27.442277 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:49:27.442278 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:49:27.442352 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:49:27.442776 (+ 424us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:49:27.442781 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:49:27.444329 (+ 1548us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:49:27.446148 (+ 1819us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:49:27.446150 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:49:27.446150 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:49:27.446398 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:49:27.446616 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:49:27.446617 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:49:27.446618 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:49:27.446690 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:49:27.446980 (+ 290us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:49:27.446981 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:49:27.447029 (+ 48us) client_negotiation.cc:769] Sending connection context
0806 03:49:27.447116 (+ 87us) client_negotiation.cc:240] Negotiation successful
0806 03:49:27.447222 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260806 03:49:27.448518 31517 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.253.252" port: 44101 } has no permanent_uuid. Determining permanent_uuid...
I20260806 03:49:27.448879 31522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:49:27.448765 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.252:44101 (local address 127.0.0.1:59768)
0806 03:49:27.448811 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0806 03:49:27.448858 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.253.252:44101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260806 03:49:27.449034 31436 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.11.253.252:44101: connect: Connection refused (error 111)
I20260806 03:49:27.449465 31522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:49:27.449378 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.252:44101 (local address 127.0.0.1:59772)
0806 03:49:27.449420 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0806 03:49:27.449447 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.253.252:44101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260806 03:49:27.449635 31517 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.253.252:44101: Network error: Client connection negotiation failed: client connection to 127.11.253.252:44101: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.253.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.252.keytab.
Entry for principal kudu/127.11.253.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.252.keytab.
I20260806 03:49:27.457175 12279 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskToVWRe/build/release/bin/kudu
/tmp/dist-test-taskToVWRe/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785988004637995-12279-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785988004637995-12279-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785988004637995-12279-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785988004637995-12279-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.253.252:44101
--webserver_interface=127.11.253.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.252.keytab
--principal=kudu/127.11.253.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.253.212:40029
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.253.254:42495,127.11.253.253:35689,127.11.253.252:44101
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785988004637995-12279-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785988004637995-12279-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785988004637995-12279-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 03:49:27.488911 31403 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.253.253" port: 35689 } attempt: 3
Aug 06 03:49:27 dist-test-slave-6ntk krb5kdc[31291](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988167, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.253.254@KRBTEST.COM for kudu/127.11.253.253@KRBTEST.COM
I20260806 03:49:27.496403 31524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:49:27.489408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42882 (local address 127.11.253.253:35689)
0806 03:49:27.489477 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0806 03:49:27.489479 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:49:27.489490 (+ 11us) server_negotiation.cc:407] Connection header received
0806 03:49:27.489541 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:49:27.489542 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:49:27.489554 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:49:27.489615 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:49:27.490013 (+ 398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:49:27.490534 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:49:27.491274 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:49:27.491558 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:49:27.491644 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:49:27.493691 (+ 2047us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:49:27.493709 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:49:27.493709 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:49:27.493717 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:49:27.495259 (+ 1542us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:49:27.495604 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:49:27.495605 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:49:27.495606 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:49:27.495667 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:49:27.495877 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:49:27.495878 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:49:27.495878 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:49:27.495993 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:49:27.496062 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0806 03:49:27.496231 (+ 169us) server_negotiation.cc:299] Negotiation successful
0806 03:49:27.496307 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260806 03:49:27.496403 31407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:49:27.489330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.253:35689 (local address 127.0.0.1:42882)
0806 03:49:27.489395 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0806 03:49:27.489406 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 03:49:27.489477 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:49:27.489633 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:49:27.489634 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:49:27.489811 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:49:27.489925 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:49:27.489931 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:49:27.490626 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:49:27.490627 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:49:27.491148 (+ 521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:49:27.491150 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:49:27.491646 (+ 496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:49:27.491647 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:49:27.491737 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:49:27.492234 (+ 497us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:49:27.492244 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:49:27.493558 (+ 1314us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:49:27.495354 (+ 1796us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:49:27.495355 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:49:27.495355 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:49:27.495530 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:49:27.495740 (+ 210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:49:27.495741 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:49:27.495742 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:49:27.495804 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:49:27.496065 (+ 261us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:49:27.496066 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:49:27.496112 (+ 46us) client_negotiation.cc:769] Sending connection context
0806 03:49:27.496190 (+ 78us) client_negotiation.cc:240] Negotiation successful
0806 03:49:27.496304 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260806 03:49:27.497026 31403 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.253.252" port: 44101 } has no permanent_uuid. Determining permanent_uuid...
I20260806 03:49:27.497421 31407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:49:27.497309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.252:44101 (local address 127.0.0.1:59784)
0806 03:49:27.497362 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0806 03:49:27.497391 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.253.252:44101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260806 03:49:27.497813 31407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:49:27.497730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.252:44101 (local address 127.0.0.1:59796)
0806 03:49:27.497775 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0806 03:49:27.497798 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.253.252:44101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260806 03:49:27.497937 31403 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.253.252:44101: Network error: Client connection negotiation failed: client connection to 127.11.253.252:44101: connect: Connection refused (error 111)
I20260806 03:49:27.498855 31517 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.253.252" port: 44101 } attempt: 1
I20260806 03:49:27.499195 31522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:49:27.499094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.252:44101 (local address 127.0.0.1:59812)
0806 03:49:27.499146 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0806 03:49:27.499174 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.253.252:44101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260806 03:49:27.499492 31522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:49:27.499426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.252:44101 (local address 127.0.0.1:59816)
0806 03:49:27.499460 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0806 03:49:27.499479 (+ 19us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.253.252:44101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
W20260806 03:49:27.499590 31517 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.253.252:44101: Network error: Client connection negotiation failed: client connection to 127.11.253.252:44101: connect: Connection refused (error 111)
I20260806 03:49:27.529137 31403 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.253.252" port: 44101 } attempt: 1
I20260806 03:49:27.529685 31407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:49:27.529541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.252:44101 (local address 127.0.0.1:59818)
0806 03:49:27.529614 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0806 03:49:27.529654 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.253.252:44101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260806 03:49:27.530094 31407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:49:27.529992 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.252:44101 (local address 127.0.0.1:59830)
0806 03:49:27.530042 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0806 03:49:27.530069 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.253.252:44101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260806 03:49:27.530216 31403 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.253.252:44101: Network error: Client connection negotiation failed: client connection to 127.11.253.252:44101: connect: Connection refused (error 111)
W20260806 03:49:27.543278 31527 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:49:27.543531 31527 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:49:27.543568 31527 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:49:27.544718 31527 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 03:49:27.544785 31527 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:49:27.544808 31527 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:49:27.544831 31527 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 03:49:27.544855 31527 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 03:49:27.546443 31527 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.253.212:40029
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785988004637995-12279-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785988004637995-12279-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.11.253.254:42495,127.11.253.253:35689,127.11.253.252:44101
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.252.keytab
--principal=kudu/127.11.253.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.253.252:44101
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785988004637995-12279-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.11.253.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31527
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785988004637995-12279-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:49:27.546722 31527 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:49:27.546967 31527 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:49:27.552105 31539 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:49:27.552150 31537 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:49:27.552181 31534 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:49:27.552515 31527 server_base.cc:1034] running on GCE node
I20260806 03:49:27.552830 31527 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:49:27.553181 31527 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:49:27.554380 31527 hybrid_clock.cc:648] HybridClock initialized: now 1785988167554351 us; error 39 us; skew 500 ppm
Aug 06 03:49:27 dist-test-slave-6ntk krb5kdc[31291](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988167, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.253.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:49:27.557080 31527 init.cc:377] Logged in from keytab as kudu/127.11.253.252@KRBTEST.COM (short username kudu)
I20260806 03:49:27.557634 31527 webserver.cc:460] Webserver started at http://127.11.253.252:37063/ using document root <none> and password file <none>
I20260806 03:49:27.557956 31527 fs_manager.cc:359] Metadata directory not provided
I20260806 03:49:27.558038 31527 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:49:27.558213 31527 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:49:27.559414 31527 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785988004637995-12279-0/minicluster-data/master-2/data/instance:
uuid: "1d6dda3bf59b4d79a089e96bbd0b0774"
format_stamp: "Formatted at 2026-08-06 03:49:27 on dist-test-slave-6ntk"
server_key: "743f18d6e40a491888663efab74f6a5b"
server_key_iv: "8ee48045b947ec5412ffcb5d748fbd3d"
server_key_version: "encryptionkey@0"
I20260806 03:49:27.559855 31527 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785988004637995-12279-0/minicluster-data/master-2/wal/instance:
uuid: "1d6dda3bf59b4d79a089e96bbd0b0774"
format_stamp: "Formatted at 2026-08-06 03:49:27 on dist-test-slave-6ntk"
server_key: "743f18d6e40a491888663efab74f6a5b"
server_key_iv: "8ee48045b947ec5412ffcb5d748fbd3d"
server_key_version: "encryptionkey@0"
I20260806 03:49:27.561425 31527 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 03:49:27.562355 31550 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:49:27.562587 31527 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260806 03:49:27.562718 31527 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785988004637995-12279-0/minicluster-data/master-2/data,/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785988004637995-12279-0/minicluster-data/master-2/wal
uuid: "1d6dda3bf59b4d79a089e96bbd0b0774"
format_stamp: "Formatted at 2026-08-06 03:49:27 on dist-test-slave-6ntk"
server_key: "743f18d6e40a491888663efab74f6a5b"
server_key_iv: "8ee48045b947ec5412ffcb5d748fbd3d"
server_key_version: "encryptionkey@0"
I20260806 03:49:27.562817 31527 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785988004637995-12279-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785988004637995-12279-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785988004637995-12279-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:49:27.567728 31517 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.253.252" port: 44101 } attempt: 2
I20260806 03:49:27.568270 31522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:49:27.568133 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.252:44101 (local address 127.0.0.1:59832)
0806 03:49:27.568196 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0806 03:49:27.568231 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.253.252:44101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260806 03:49:27.568676 31522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:49:27.568542 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.252:44101 (local address 127.0.0.1:59834)
0806 03:49:27.568600 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0806 03:49:27.568644 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.253.252:44101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260806 03:49:27.568818 31517 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.253.252:44101: Network error: Client connection negotiation failed: client connection to 127.11.253.252:44101: connect: Connection refused (error 111)
I20260806 03:49:27.575464 31527 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:49:27.577677 31527 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:49:27.577935 31527 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:49:27.581962 31527 rpc_server.cc:307] RPC server started. Bound to: 127.11.253.252:44101
I20260806 03:49:27.582022 31626 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.253.252:44101 every 8 connection(s)
I20260806 03:49:27.582352 31527 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785988004637995-12279-0/minicluster-data/master-2/data/info.pb
I20260806 03:49:27.582734 12279 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskToVWRe/build/release/bin/kudu as pid 31527
I20260806 03:49:27.582873 12279 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785988004637995-12279-0/minicluster-data/master-2/wal/instance
I20260806 03:49:27.583062 12279 external_mini_cluster.cc:1428] Setting key 5e1532fcce206332a24c14d09d654071
I20260806 03:49:27.583369 31627 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:49:27.585923 31627 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.253.254" port: 42495 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 03:49:27 dist-test-slave-6ntk krb5kdc[31291](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988167, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.253.252@KRBTEST.COM
Aug 06 03:49:27 dist-test-slave-6ntk krb5kdc[31291](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988167, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.253.252@KRBTEST.COM for kudu/127.11.253.254@KRBTEST.COM
I20260806 03:49:27.593993 31637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:49:27.583559 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59850 (local address 127.11.253.252:44101)
0806 03:49:27.583744 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0806 03:49:27.583753 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 03:49:27.583791 (+ 38us) server_negotiation.cc:407] Connection header received
0806 03:49:27.584270 (+ 479us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:49:27.584277 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:49:27.584327 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:49:27.584448 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:49:27.585270 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:49:27.586381 (+ 1111us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:49:27.587204 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:49:27.587779 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:49:27.587855 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:49:27.590599 (+ 2744us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:49:27.590608 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:49:27.590608 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:49:27.590616 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:49:27.592320 (+ 1704us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:49:27.592684 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:49:27.592687 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:49:27.592687 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:49:27.592780 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:49:27.592987 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:49:27.592988 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:49:27.592989 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:49:27.593335 (+ 346us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:49:27.593519 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0806 03:49:27.593590 (+ 71us) server_negotiation.cc:299] Negotiation successful
0806 03:49:27.593727 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":102,"threads_started":1}
I20260806 03:49:27.594506 31406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:49:27.586628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39942 (local address 127.11.253.254:42495)
0806 03:49:27.586672 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0806 03:49:27.586674 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:49:27.586836 (+ 162us) server_negotiation.cc:407] Connection header received
0806 03:49:27.586914 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:49:27.586915 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:49:27.586926 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:49:27.586985 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:49:27.587530 (+ 545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:49:27.588132 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:49:27.588963 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:49:27.589284 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:49:27.589355 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:49:27.591695 (+ 2340us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:49:27.591696 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:49:27.591697 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:49:27.591705 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:49:27.593143 (+ 1438us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:49:27.593565 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:49:27.593566 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:49:27.593566 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:49:27.593626 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:49:27.593854 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:49:27.593855 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:49:27.593855 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:49:27.594018 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:49:27.594103 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0806 03:49:27.594340 (+ 237us) server_negotiation.cc:299] Negotiation successful
0806 03:49:27.594417 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260806 03:49:27.594520 31638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:49:27.586576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:42495 (local address 127.0.0.1:39942)
0806 03:49:27.586741 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0806 03:49:27.586774 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0806 03:49:27.586852 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:49:27.587004 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:49:27.587007 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:49:27.587363 (+ 356us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:49:27.587436 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:49:27.587438 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:49:27.588233 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:49:27.588234 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:49:27.588872 (+ 638us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:49:27.588874 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:49:27.589384 (+ 510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:49:27.589385 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:49:27.589465 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:49:27.589910 (+ 445us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:49:27.589920 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:49:27.591626 (+ 1706us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:49:27.593219 (+ 1593us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:49:27.593225 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:49:27.593228 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:49:27.593486 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:49:27.593712 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:49:27.593712 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:49:27.593713 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:49:27.593774 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:49:27.594138 (+ 364us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:49:27.594146 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:49:27.594198 (+ 52us) client_negotiation.cc:769] Sending connection context
0806 03:49:27.594305 (+ 107us) client_negotiation.cc:240] Negotiation successful
0806 03:49:27.594417 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":81,"threads_started":1}
I20260806 03:49:27.595044 31627 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.253.253" port: 35689 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 03:49:27 dist-test-slave-6ntk krb5kdc[31291](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988167, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.253.252@KRBTEST.COM for kudu/127.11.253.253@KRBTEST.COM
Aug 06 03:49:27 dist-test-slave-6ntk krb5kdc[31291](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988167, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.253.254@KRBTEST.COM
I20260806 03:49:27.602044 31638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:49:27.595303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.253:35689 (local address 127.0.0.1:42890)
0806 03:49:27.595343 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0806 03:49:27.595353 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 03:49:27.595385 (+ 32us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:49:27.595589 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:49:27.595590 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:49:27.595750 (+ 160us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:49:27.595781 (+ 31us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:49:27.595783 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:49:27.596470 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:49:27.596471 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:49:27.596938 (+ 467us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:49:27.596940 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:49:27.597344 (+ 404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:49:27.597345 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:49:27.597400 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:49:27.597820 (+ 420us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:49:27.597826 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:49:27.599066 (+ 1240us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:49:27.601007 (+ 1941us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:49:27.601008 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:49:27.601008 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:49:27.601176 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:49:27.601391 (+ 215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:49:27.601392 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:49:27.601392 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:49:27.601454 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:49:27.601706 (+ 252us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:49:27.601707 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:49:27.601747 (+ 40us) client_negotiation.cc:769] Sending connection context
0806 03:49:27.601832 (+ 85us) client_negotiation.cc:240] Negotiation successful
0806 03:49:27.601945 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27}
I20260806 03:49:27.602093 31524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:49:27.595397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42890 (local address 127.11.253.253:35689)
0806 03:49:27.595450 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0806 03:49:27.595452 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:49:27.595461 (+ 9us) server_negotiation.cc:407] Connection header received
0806 03:49:27.595486 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:49:27.595487 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:49:27.595498 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:49:27.595564 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:49:27.595860 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:49:27.596368 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:49:27.597054 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:49:27.597267 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:49:27.597345 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:49:27.599168 (+ 1823us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:49:27.599170 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:49:27.599170 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:49:27.599177 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:49:27.600884 (+ 1707us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:49:27.601253 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:49:27.601254 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:49:27.601254 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:49:27.601314 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:49:27.601518 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:49:27.601519 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:49:27.601519 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:49:27.601631 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:49:27.601702 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0806 03:49:27.601921 (+ 219us) server_negotiation.cc:299] Negotiation successful
0806 03:49:27.601996 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260806 03:49:27.602464 31406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:49:27.594873 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39950 (local address 127.11.253.254:42495)
0806 03:49:27.594907 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0806 03:49:27.594909 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:49:27.594916 (+ 7us) server_negotiation.cc:407] Connection header received
0806 03:49:27.594933 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:49:27.594933 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:49:27.594942 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:49:27.594997 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:49:27.595601 (+ 604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:49:27.596147 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:49:27.597055 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:49:27.597276 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:49:27.597358 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:49:27.599735 (+ 2377us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:49:27.599736 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:49:27.599737 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:49:27.599744 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:49:27.601189 (+ 1445us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:49:27.601622 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:49:27.601623 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:49:27.601624 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:49:27.601681 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:49:27.601927 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:49:27.601928 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:49:27.601928 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:49:27.602034 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:49:27.602252 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0806 03:49:27.602289 (+ 37us) server_negotiation.cc:299] Negotiation successful
0806 03:49:27.602366 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260806 03:49:27.602834 31627 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.253.252" port: 44101 } has no permanent_uuid. Determining permanent_uuid...
I20260806 03:49:27.605423 31403 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.253.252" port: 44101 } attempt: 2
Aug 06 03:49:27 dist-test-slave-6ntk krb5kdc[31291](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988167, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.253.253@KRBTEST.COM
Aug 06 03:49:27 dist-test-slave-6ntk krb5kdc[31291](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988167, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.253.252@KRBTEST.COM for kudu/127.11.253.252@KRBTEST.COM
Aug 06 03:49:27 dist-test-slave-6ntk krb5kdc[31291](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988167, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.253.254@KRBTEST.COM for kudu/127.11.253.252@KRBTEST.COM
I20260806 03:49:27.611949 31638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:49:27.603119 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.252:44101 (local address 127.0.0.1:59852)
0806 03:49:27.603192 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0806 03:49:27.603200 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0806 03:49:27.603270 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:49:27.603391 (+ 121us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:49:27.603392 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:49:27.603522 (+ 130us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:49:27.603544 (+ 22us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:49:27.603546 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:49:27.604152 (+ 606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:49:27.604153 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:49:27.604613 (+ 460us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:49:27.604614 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:49:27.604996 (+ 382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:49:27.604997 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:49:27.605044 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:49:27.605671 (+ 627us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:49:27.605676 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:49:27.608903 (+ 3227us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:49:27.610828 (+ 1925us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:49:27.610829 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:49:27.610829 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:49:27.611055 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:49:27.611232 (+ 177us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:49:27.611232 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:49:27.611233 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:49:27.611299 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:49:27.611542 (+ 243us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:49:27.611543 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:49:27.611580 (+ 37us) client_negotiation.cc:769] Sending connection context
0806 03:49:27.611664 (+ 84us) client_negotiation.cc:240] Negotiation successful
0806 03:49:27.611835 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":66}
I20260806 03:49:27.612145 31524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:49:27.603143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42896 (local address 127.11.253.253:35689)
0806 03:49:27.603185 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0806 03:49:27.603187 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:49:27.603225 (+ 38us) server_negotiation.cc:407] Connection header received
0806 03:49:27.603294 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:49:27.603295 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:49:27.603305 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:49:27.603359 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:49:27.603809 (+ 450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:49:27.604242 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:49:27.604876 (+ 634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:49:27.605072 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:49:27.605145 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:49:27.608828 (+ 3683us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:49:27.608829 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:49:27.608830 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:49:27.608838 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:49:27.610674 (+ 1836us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:49:27.611055 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:49:27.611056 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:49:27.611057 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:49:27.611117 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:49:27.611349 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:49:27.611350 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:49:27.611351 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:49:27.611489 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:49:27.611687 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0806 03:49:27.611944 (+ 257us) server_negotiation.cc:299] Negotiation successful
0806 03:49:27.612022 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260806 03:49:27.611995 31637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:49:27.603144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59852 (local address 127.11.253.252:44101)
0806 03:49:27.603175 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0806 03:49:27.603176 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 03:49:27.603256 (+ 80us) server_negotiation.cc:407] Connection header received
0806 03:49:27.603324 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:49:27.603325 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:49:27.603333 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:49:27.603388 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:49:27.603625 (+ 237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:49:27.604074 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:49:27.604694 (+ 620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:49:27.604909 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:49:27.604995 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:49:27.608992 (+ 3997us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:49:27.608994 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:49:27.608994 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:49:27.609001 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:49:27.610746 (+ 1745us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:49:27.611124 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:49:27.611125 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:49:27.611125 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:49:27.611178 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:49:27.611350 (+ 172us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:49:27.611351 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:49:27.611352 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:49:27.611462 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:49:27.611537 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0806 03:49:27.611686 (+ 149us) server_negotiation.cc:299] Negotiation successful
0806 03:49:27.611890 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":136,"server-negotiator.queue_time_us":16}
I20260806 03:49:27.614373 12279 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 03:49:27.614486 31627 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1d6dda3bf59b4d79a089e96bbd0b0774: Bootstrap starting.
I20260806 03:49:27.615015 31407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:49:27.606141 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.252:44101 (local address 127.0.0.1:59864)
0806 03:49:27.606200 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0806 03:49:27.606209 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 03:49:27.606243 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:49:27.606676 (+ 433us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:49:27.606677 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:49:27.606935 (+ 258us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:49:27.606975 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:49:27.606978 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:49:27.608521 (+ 1543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:49:27.608522 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:49:27.609303 (+ 781us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:49:27.609305 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:49:27.609858 (+ 553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:49:27.609859 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:49:27.609961 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:49:27.610246 (+ 285us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:49:27.610251 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:49:27.611563 (+ 1312us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:49:27.613735 (+ 2172us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:49:27.613736 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:49:27.613736 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:49:27.614029 (+ 293us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:49:27.614295 (+ 266us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:49:27.614296 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:49:27.614297 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:49:27.614368 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:49:27.614674 (+ 306us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:49:27.614675 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:49:27.614722 (+ 47us) client_negotiation.cc:769] Sending connection context
0806 03:49:27.614800 (+ 78us) client_negotiation.cc:240] Negotiation successful
0806 03:49:27.614923 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260806 03:49:27.615085 31640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:49:27.606332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59864 (local address 127.11.253.252:44101)
0806 03:49:27.606528 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0806 03:49:27.606530 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:49:27.606539 (+ 9us) server_negotiation.cc:407] Connection header received
0806 03:49:27.606558 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:49:27.606559 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:49:27.606569 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:49:27.606640 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:49:27.607135 (+ 495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:49:27.608432 (+ 1297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:49:27.609399 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:49:27.609762 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:49:27.609848 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:49:27.611651 (+ 1803us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:49:27.611653 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:49:27.611653 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:49:27.611661 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:49:27.613597 (+ 1936us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:49:27.614086 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:49:27.614087 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:49:27.614088 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:49:27.614151 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:49:27.614439 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:49:27.614440 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:49:27.614440 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:49:27.614597 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:49:27.614660 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0806 03:49:27.614839 (+ 179us) server_negotiation.cc:299] Negotiation successful
0806 03:49:27.614934 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":154,"thread_start_us":97,"threads_started":1}
I20260806 03:49:27.615278 31627 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1d6dda3bf59b4d79a089e96bbd0b0774: Neither blocks nor log segments found. Creating new log.
I20260806 03:49:27.615625 31627 log.cc:824] T 00000000000000000000000000000000 P 1d6dda3bf59b4d79a089e96bbd0b0774: Log is configured to *not* fsync() on all Append() calls
I20260806 03:49:27.616549 31627 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1d6dda3bf59b4d79a089e96bbd0b0774: No bootstrap required, opened a new log
I20260806 03:49:27.617125 31403 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8400f5fc4fda4d069125f49070b47383: Bootstrap starting.
I20260806 03:49:27.617812 31403 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8400f5fc4fda4d069125f49070b47383: Neither blocks nor log segments found. Creating new log.
I20260806 03:49:27.618144 31403 log.cc:824] T 00000000000000000000000000000000 P 8400f5fc4fda4d069125f49070b47383: Log is configured to *not* fsync() on all Append() calls
I20260806 03:49:27.618880 31627 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1d6dda3bf59b4d79a089e96bbd0b0774 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8400f5fc4fda4d069125f49070b47383" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 42495 } } peers { permanent_uuid: "e6f42db43a2b4a3dbf9bf81900be56c3" member_type: VOTER last_known_addr { host: "127.11.253.253" port: 35689 } } peers { permanent_uuid: "1d6dda3bf59b4d79a089e96bbd0b0774" member_type: VOTER last_known_addr { host: "127.11.253.252" port: 44101 } }
I20260806 03:49:27.618986 31403 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8400f5fc4fda4d069125f49070b47383: No bootstrap required, opened a new log
I20260806 03:49:27.619030 31627 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1d6dda3bf59b4d79a089e96bbd0b0774 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:49:27.619094 31627 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1d6dda3bf59b4d79a089e96bbd0b0774 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1d6dda3bf59b4d79a089e96bbd0b0774, State: Initialized, Role: FOLLOWER
I20260806 03:49:27.619230 31627 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1d6dda3bf59b4d79a089e96bbd0b0774 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8400f5fc4fda4d069125f49070b47383" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 42495 } } peers { permanent_uuid: "e6f42db43a2b4a3dbf9bf81900be56c3" member_type: VOTER last_known_addr { host: "127.11.253.253" port: 35689 } } peers { permanent_uuid: "1d6dda3bf59b4d79a089e96bbd0b0774" member_type: VOTER last_known_addr { host: "127.11.253.252" port: 44101 } }
I20260806 03:49:27.619563 31627 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1d6dda3bf59b4d79a089e96bbd0b0774 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 03:49:27.621140 31660 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1d6dda3bf59b4d79a089e96bbd0b0774 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8400f5fc4fda4d069125f49070b47383" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 42495 } } peers { permanent_uuid: "e6f42db43a2b4a3dbf9bf81900be56c3" member_type: VOTER last_known_addr { host: "127.11.253.253" port: 35689 } } peers { permanent_uuid: "1d6dda3bf59b4d79a089e96bbd0b0774" member_type: VOTER last_known_addr { host: "127.11.253.252" port: 44101 } } }
I20260806 03:49:27.621317 31660 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1d6dda3bf59b4d79a089e96bbd0b0774 [sys.catalog]: This master's current role is: FOLLOWER
W20260806 03:49:27.621414 31674 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1d6dda3bf59b4d79a089e96bbd0b0774: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 03:49:27.621500 31674 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 03:49:27.624379 31640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:49:27.616260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59870 (local address 127.11.253.252:44101)
0806 03:49:27.616567 (+ 307us) server_negotiation.cc:206] Beginning negotiation
0806 03:49:27.616569 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:49:27.616579 (+ 10us) server_negotiation.cc:407] Connection header received
0806 03:49:27.616742 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:49:27.616743 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:49:27.616758 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:49:27.616800 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:49:27.617581 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:49:27.618366 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:49:27.619273 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:49:27.619561 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:49:27.619638 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:49:27.621366 (+ 1728us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:49:27.621368 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:49:27.621368 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:49:27.621376 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:49:27.623102 (+ 1726us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:49:27.623530 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:49:27.623531 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:49:27.623532 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:49:27.623587 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:49:27.623781 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:49:27.623782 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:49:27.623782 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:49:27.623908 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:49:27.623981 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0806 03:49:27.624197 (+ 216us) server_negotiation.cc:299] Negotiation successful
0806 03:49:27.624280 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":288}
I20260806 03:49:27.621995 31527 master_runner.cc:186] Error getting master registration for 127.11.253.254:42495: OK, instance_id { permanent_uuid: "8400f5fc4fda4d069125f49070b47383" instance_seqno: 1785988167260611 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 03:49:27.624827 31527 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 03:49:27.625861 31403 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8400f5fc4fda4d069125f49070b47383 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8400f5fc4fda4d069125f49070b47383" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 42495 } } peers { permanent_uuid: "e6f42db43a2b4a3dbf9bf81900be56c3" member_type: VOTER last_known_addr { host: "127.11.253.253" port: 35689 } } peers { permanent_uuid: "1d6dda3bf59b4d79a089e96bbd0b0774" member_type: VOTER last_known_addr { host: "127.11.253.252" port: 44101 } }
I20260806 03:49:27.626773 31403 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8400f5fc4fda4d069125f49070b47383 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:49:27.626861 31403 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8400f5fc4fda4d069125f49070b47383 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8400f5fc4fda4d069125f49070b47383, State: Initialized, Role: FOLLOWER
I20260806 03:49:27.627020 31403 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8400f5fc4fda4d069125f49070b47383 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8400f5fc4fda4d069125f49070b47383" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 42495 } } peers { permanent_uuid: "e6f42db43a2b4a3dbf9bf81900be56c3" member_type: VOTER last_known_addr { host: "127.11.253.253" port: 35689 } } peers { permanent_uuid: "1d6dda3bf59b4d79a089e96bbd0b0774" member_type: VOTER last_known_addr { host: "127.11.253.252" port: 44101 } }
I20260806 03:49:27.627732 31403 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8400f5fc4fda4d069125f49070b47383 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 03:49:27.628674 31678 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8400f5fc4fda4d069125f49070b47383 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8400f5fc4fda4d069125f49070b47383" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 42495 } } peers { permanent_uuid: "e6f42db43a2b4a3dbf9bf81900be56c3" member_type: VOTER last_known_addr { host: "127.11.253.253" port: 35689 } } peers { permanent_uuid: "1d6dda3bf59b4d79a089e96bbd0b0774" member_type: VOTER last_known_addr { host: "127.11.253.252" port: 44101 } } }
I20260806 03:49:27.629087 31678 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8400f5fc4fda4d069125f49070b47383 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 03:49:27.631508 31524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:49:27.616875 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42898 (local address 127.11.253.253:35689)
0806 03:49:27.616918 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0806 03:49:27.616920 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:49:27.621669 (+ 4749us) server_negotiation.cc:407] Connection header received
0806 03:49:27.621736 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:49:27.621737 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:49:27.621749 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:49:27.621802 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:49:27.623037 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:49:27.623717 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:49:27.625644 (+ 1927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:49:27.625983 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:49:27.626054 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:49:27.628050 (+ 1996us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:49:27.628052 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:49:27.628052 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:49:27.628059 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:49:27.629961 (+ 1902us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:49:27.630582 (+ 621us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:49:27.630584 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:49:27.630584 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:49:27.630648 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:49:27.630870 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:49:27.630871 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:49:27.630872 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:49:27.631006 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:49:27.631074 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0806 03:49:27.631310 (+ 236us) server_negotiation.cc:299] Negotiation successful
0806 03:49:27.631390 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260806 03:49:27.631560 31406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:49:27.616581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39962 (local address 127.11.253.254:42495)
0806 03:49:27.616627 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0806 03:49:27.616629 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:49:27.621552 (+ 4923us) server_negotiation.cc:407] Connection header received
0806 03:49:27.621822 (+ 270us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:49:27.621823 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:49:27.621834 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:49:27.621863 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:49:27.623826 (+ 1963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:49:27.624529 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:49:27.626584 (+ 2055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:49:27.626890 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:49:27.626963 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:49:27.628296 (+ 1333us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:49:27.628298 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:49:27.628298 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:49:27.628305 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:49:27.629964 (+ 1659us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:49:27.630547 (+ 583us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:49:27.630549 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:49:27.630549 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:49:27.630619 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:49:27.630825 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:49:27.630826 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:49:27.630826 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:49:27.630956 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:49:27.631044 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0806 03:49:27.631391 (+ 347us) server_negotiation.cc:299] Negotiation successful
0806 03:49:27.631472 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260806 03:49:27.631614 31305 master_runner.cc:186] Error getting master registration for 127.11.253.253:35689: OK, instance_id { permanent_uuid: "e6f42db43a2b4a3dbf9bf81900be56c3" instance_seqno: 1785988167423369 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 03:49:27.633956 31305 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260806 03:49:27.634754 31698 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 8400f5fc4fda4d069125f49070b47383: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 03:49:27.634817 31698 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 03:49:27.659991 31517 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.253.252" port: 44101 } attempt: 3
Aug 06 03:49:27 dist-test-slave-6ntk krb5kdc[31291](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988167, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.253.253@KRBTEST.COM for kudu/127.11.253.252@KRBTEST.COM
I20260806 03:49:27.668607 31640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:49:27.660529 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59880 (local address 127.11.253.252:44101)
0806 03:49:27.660589 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0806 03:49:27.660591 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:49:27.660606 (+ 15us) server_negotiation.cc:407] Connection header received
0806 03:49:27.660695 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:49:27.660696 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:49:27.660708 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:49:27.660789 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:49:27.661146 (+ 357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:49:27.661952 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:49:27.662774 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:49:27.663053 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:49:27.663137 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:49:27.665298 (+ 2161us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:49:27.665300 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:49:27.665300 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:49:27.665308 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:49:27.667322 (+ 2014us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:49:27.667718 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:49:27.667720 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:49:27.667720 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:49:27.667789 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:49:27.668013 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:49:27.668014 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:49:27.668015 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:49:27.668146 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:49:27.668231 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0806 03:49:27.668401 (+ 170us) server_negotiation.cc:299] Negotiation successful
0806 03:49:27.668492 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260806 03:49:27.668607 31522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:49:27.660489 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.252:44101 (local address 127.0.0.1:59880)
0806 03:49:27.660556 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0806 03:49:27.660569 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 03:49:27.660621 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:49:27.660821 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:49:27.660822 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:49:27.661008 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:49:27.661055 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:49:27.661057 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:49:27.662047 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:49:27.662048 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:49:27.662675 (+ 627us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:49:27.662677 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:49:27.663136 (+ 459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:49:27.663137 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:49:27.663196 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:49:27.663770 (+ 574us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:49:27.663776 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:49:27.665175 (+ 1399us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:49:27.667411 (+ 2236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:49:27.667412 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:49:27.667413 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:49:27.667618 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:49:27.667865 (+ 247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:49:27.667865 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:49:27.667866 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:49:27.667940 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:49:27.668222 (+ 282us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:49:27.668223 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:49:27.668275 (+ 52us) client_negotiation.cc:769] Sending connection context
0806 03:49:27.668374 (+ 99us) client_negotiation.cc:240] Negotiation successful
0806 03:49:27.668492 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42}
I20260806 03:49:27.670686 31517 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e6f42db43a2b4a3dbf9bf81900be56c3: Bootstrap starting.
I20260806 03:49:27.671516 31517 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e6f42db43a2b4a3dbf9bf81900be56c3: Neither blocks nor log segments found. Creating new log.
I20260806 03:49:27.671869 31517 log.cc:824] T 00000000000000000000000000000000 P e6f42db43a2b4a3dbf9bf81900be56c3: Log is configured to *not* fsync() on all Append() calls
I20260806 03:49:27.672830 31517 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e6f42db43a2b4a3dbf9bf81900be56c3: No bootstrap required, opened a new log
I20260806 03:49:27.674978 31517 raft_consensus.cc:348] T 00000000000000000000000000000000 P e6f42db43a2b4a3dbf9bf81900be56c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8400f5fc4fda4d069125f49070b47383" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 42495 } } peers { permanent_uuid: "e6f42db43a2b4a3dbf9bf81900be56c3" member_type: VOTER last_known_addr { host: "127.11.253.253" port: 35689 } } peers { permanent_uuid: "1d6dda3bf59b4d79a089e96bbd0b0774" member_type: VOTER last_known_addr { host: "127.11.253.252" port: 44101 } }
I20260806 03:49:27.675268 31517 raft_consensus.cc:374] T 00000000000000000000000000000000 P e6f42db43a2b4a3dbf9bf81900be56c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:49:27.675372 31517 raft_consensus.cc:729] T 00000000000000000000000000000000 P e6f42db43a2b4a3dbf9bf81900be56c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6f42db43a2b4a3dbf9bf81900be56c3, State: Initialized, Role: FOLLOWER
I20260806 03:49:27.675526 31517 consensus_queue.cc:260] T 00000000000000000000000000000000 P e6f42db43a2b4a3dbf9bf81900be56c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8400f5fc4fda4d069125f49070b47383" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 42495 } } peers { permanent_uuid: "e6f42db43a2b4a3dbf9bf81900be56c3" member_type: VOTER last_known_addr { host: "127.11.253.253" port: 35689 } } peers { permanent_uuid: "1d6dda3bf59b4d79a089e96bbd0b0774" member_type: VOTER last_known_addr { host: "127.11.253.252" port: 44101 } }
I20260806 03:49:27.675899 31704 sys_catalog.cc:455] T 00000000000000000000000000000000 P e6f42db43a2b4a3dbf9bf81900be56c3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8400f5fc4fda4d069125f49070b47383" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 42495 } } peers { permanent_uuid: "e6f42db43a2b4a3dbf9bf81900be56c3" member_type: VOTER last_known_addr { host: "127.11.253.253" port: 35689 } } peers { permanent_uuid: "1d6dda3bf59b4d79a089e96bbd0b0774" member_type: VOTER last_known_addr { host: "127.11.253.252" port: 44101 } } }
I20260806 03:49:27.675973 31517 sys_catalog.cc:564] T 00000000000000000000000000000000 P e6f42db43a2b4a3dbf9bf81900be56c3 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 03:49:27.676168 31704 sys_catalog.cc:458] T 00000000000000000000000000000000 P e6f42db43a2b4a3dbf9bf81900be56c3 [sys.catalog]: This master's current role is: FOLLOWER
W20260806 03:49:27.678236 31723 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e6f42db43a2b4a3dbf9bf81900be56c3: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 03:49:27.678373 31723 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 03:49:27.680668 31411 master_runner.cc:263] No leader master found from master e6f42db43a2b4a3dbf9bf81900be56c3
I20260806 03:49:27.680727 31411 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260806 03:49:27.758499 31330 tablet.cc:2369] T 00000000000000000000000000000000 P 8400f5fc4fda4d069125f49070b47383: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 03:49:27.803495 31678 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8400f5fc4fda4d069125f49070b47383 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 03:49:27.803653 31678 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8400f5fc4fda4d069125f49070b47383 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8400f5fc4fda4d069125f49070b47383" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 42495 } } peers { permanent_uuid: "e6f42db43a2b4a3dbf9bf81900be56c3" member_type: VOTER last_known_addr { host: "127.11.253.253" port: 35689 } } peers { permanent_uuid: "1d6dda3bf59b4d79a089e96bbd0b0774" member_type: VOTER last_known_addr { host: "127.11.253.252" port: 44101 } }
I20260806 03:49:27.803905 31678 leader_election.cc:288] T 00000000000000000000000000000000 P 8400f5fc4fda4d069125f49070b47383 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e6f42db43a2b4a3dbf9bf81900be56c3 (127.11.253.253:35689), 1d6dda3bf59b4d79a089e96bbd0b0774 (127.11.253.252:44101)
I20260806 03:49:27.804455 31475 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "8400f5fc4fda4d069125f49070b47383" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6f42db43a2b4a3dbf9bf81900be56c3" is_pre_election: true
I20260806 03:49:27.804641 31475 raft_consensus.cc:2454] T 00000000000000000000000000000000 P e6f42db43a2b4a3dbf9bf81900be56c3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8400f5fc4fda4d069125f49070b47383 in term 0.
I20260806 03:49:27.804957 31328 leader_election.cc:302] T 00000000000000000000000000000000 P 8400f5fc4fda4d069125f49070b47383 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8400f5fc4fda4d069125f49070b47383, e6f42db43a2b4a3dbf9bf81900be56c3; no voters:
I20260806 03:49:27.805189 31678 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8400f5fc4fda4d069125f49070b47383 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 03:49:27.805258 31678 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8400f5fc4fda4d069125f49070b47383 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 03:49:27.805281 31678 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8400f5fc4fda4d069125f49070b47383 [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:49:27.806350 31678 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8400f5fc4fda4d069125f49070b47383 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8400f5fc4fda4d069125f49070b47383" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 42495 } } peers { permanent_uuid: "e6f42db43a2b4a3dbf9bf81900be56c3" member_type: VOTER last_known_addr { host: "127.11.253.253" port: 35689 } } peers { permanent_uuid: "1d6dda3bf59b4d79a089e96bbd0b0774" member_type: VOTER last_known_addr { host: "127.11.253.252" port: 44101 } }
I20260806 03:49:27.806514 31678 leader_election.cc:288] T 00000000000000000000000000000000 P 8400f5fc4fda4d069125f49070b47383 [CANDIDATE]: Term 1 election: Requested vote from peers e6f42db43a2b4a3dbf9bf81900be56c3 (127.11.253.253:35689), 1d6dda3bf59b4d79a089e96bbd0b0774 (127.11.253.252:44101)
I20260806 03:49:27.806942 31475 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "8400f5fc4fda4d069125f49070b47383" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6f42db43a2b4a3dbf9bf81900be56c3"
I20260806 03:49:27.807040 31475 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e6f42db43a2b4a3dbf9bf81900be56c3 [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:49:27.807972 31475 raft_consensus.cc:2454] T 00000000000000000000000000000000 P e6f42db43a2b4a3dbf9bf81900be56c3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8400f5fc4fda4d069125f49070b47383 in term 1.
I20260806 03:49:27.808223 31328 leader_election.cc:302] T 00000000000000000000000000000000 P 8400f5fc4fda4d069125f49070b47383 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8400f5fc4fda4d069125f49070b47383, e6f42db43a2b4a3dbf9bf81900be56c3; no voters:
I20260806 03:49:27.808612 31589 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "8400f5fc4fda4d069125f49070b47383" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1d6dda3bf59b4d79a089e96bbd0b0774"
I20260806 03:49:27.808741 31589 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1d6dda3bf59b4d79a089e96bbd0b0774 [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:49:27.809516 31589 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 1d6dda3bf59b4d79a089e96bbd0b0774 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8400f5fc4fda4d069125f49070b47383 in term 1.
I20260806 03:49:27.809631 31590 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "8400f5fc4fda4d069125f49070b47383" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1d6dda3bf59b4d79a089e96bbd0b0774" is_pre_election: true
I20260806 03:49:27.809687 31590 raft_consensus.cc:2362] T 00000000000000000000000000000000 P 1d6dda3bf59b4d79a089e96bbd0b0774 [term 1 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 8400f5fc4fda4d069125f49070b47383 in term 1. Re-sending same reply.
I20260806 03:49:27.810071 31678 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8400f5fc4fda4d069125f49070b47383 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 03:49:27.810189 31678 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8400f5fc4fda4d069125f49070b47383 [term 1 LEADER]: Becoming Leader. State: Replica: 8400f5fc4fda4d069125f49070b47383, State: Running, Role: LEADER
I20260806 03:49:27.810276 31678 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8400f5fc4fda4d069125f49070b47383 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8400f5fc4fda4d069125f49070b47383" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 42495 } } peers { permanent_uuid: "e6f42db43a2b4a3dbf9bf81900be56c3" member_type: VOTER last_known_addr { host: "127.11.253.253" port: 35689 } } peers { permanent_uuid: "1d6dda3bf59b4d79a089e96bbd0b0774" member_type: VOTER last_known_addr { host: "127.11.253.252" port: 44101 } }
I20260806 03:49:27.810631 31678 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8400f5fc4fda4d069125f49070b47383 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8400f5fc4fda4d069125f49070b47383. Latest consensus state: current_term: 1 leader_uuid: "8400f5fc4fda4d069125f49070b47383" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8400f5fc4fda4d069125f49070b47383" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 42495 } } peers { permanent_uuid: "e6f42db43a2b4a3dbf9bf81900be56c3" member_type: VOTER last_known_addr { host: "127.11.253.253" port: 35689 } } peers { permanent_uuid: "1d6dda3bf59b4d79a089e96bbd0b0774" member_type: VOTER last_known_addr { host: "127.11.253.252" port: 44101 } } }
I20260806 03:49:27.810684 31678 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8400f5fc4fda4d069125f49070b47383 [sys.catalog]: This master's current role is: LEADER
I20260806 03:49:27.811834 31727 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 03:49:27.812057 31727 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 03:49:27.814173 31590 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 1d6dda3bf59b4d79a089e96bbd0b0774 [term 1 FOLLOWER]: Refusing update from remote peer 8400f5fc4fda4d069125f49070b47383: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 03:49:27.814491 31475 raft_consensus.cc:1261] T 00000000000000000000000000000000 P e6f42db43a2b4a3dbf9bf81900be56c3 [term 1 FOLLOWER]: Refusing update from remote peer 8400f5fc4fda4d069125f49070b47383: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 03:49:27.815061 31678 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 8400f5fc4fda4d069125f49070b47383 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e6f42db43a2b4a3dbf9bf81900be56c3" member_type: VOTER last_known_addr { host: "127.11.253.253" port: 35689 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 03:49:27.815197 31678 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 8400f5fc4fda4d069125f49070b47383 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1d6dda3bf59b4d79a089e96bbd0b0774" member_type: VOTER last_known_addr { host: "127.11.253.252" port: 44101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 03:49:27.816545 31704 sys_catalog.cc:455] T 00000000000000000000000000000000 P e6f42db43a2b4a3dbf9bf81900be56c3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8400f5fc4fda4d069125f49070b47383. Latest consensus state: current_term: 1 leader_uuid: "8400f5fc4fda4d069125f49070b47383" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8400f5fc4fda4d069125f49070b47383" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 42495 } } peers { permanent_uuid: "e6f42db43a2b4a3dbf9bf81900be56c3" member_type: VOTER last_known_addr { host: "127.11.253.253" port: 35689 } } peers { permanent_uuid: "1d6dda3bf59b4d79a089e96bbd0b0774" member_type: VOTER last_known_addr { host: "127.11.253.252" port: 44101 } } }
I20260806 03:49:27.816627 31704 sys_catalog.cc:458] T 00000000000000000000000000000000 P e6f42db43a2b4a3dbf9bf81900be56c3 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 03:49:27.817735 31660 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1d6dda3bf59b4d79a089e96bbd0b0774 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8400f5fc4fda4d069125f49070b47383. Latest consensus state: current_term: 1 leader_uuid: "8400f5fc4fda4d069125f49070b47383" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8400f5fc4fda4d069125f49070b47383" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 42495 } } peers { permanent_uuid: "e6f42db43a2b4a3dbf9bf81900be56c3" member_type: VOTER last_known_addr { host: "127.11.253.253" port: 35689 } } peers { permanent_uuid: "1d6dda3bf59b4d79a089e96bbd0b0774" member_type: VOTER last_known_addr { host: "127.11.253.252" port: 44101 } } }
I20260806 03:49:27.817831 31660 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1d6dda3bf59b4d79a089e96bbd0b0774 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 03:49:27.819299 31678 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8400f5fc4fda4d069125f49070b47383 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "8400f5fc4fda4d069125f49070b47383" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8400f5fc4fda4d069125f49070b47383" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 42495 } } peers { permanent_uuid: "e6f42db43a2b4a3dbf9bf81900be56c3" member_type: VOTER last_known_addr { host: "127.11.253.253" port: 35689 } } peers { permanent_uuid: "1d6dda3bf59b4d79a089e96bbd0b0774" member_type: VOTER last_known_addr { host: "127.11.253.252" port: 44101 } } }
I20260806 03:49:27.819423 31678 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8400f5fc4fda4d069125f49070b47383 [sys.catalog]: This master's current role is: LEADER
I20260806 03:49:27.819499 31704 sys_catalog.cc:455] T 00000000000000000000000000000000 P e6f42db43a2b4a3dbf9bf81900be56c3 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "8400f5fc4fda4d069125f49070b47383" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8400f5fc4fda4d069125f49070b47383" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 42495 } } peers { permanent_uuid: "e6f42db43a2b4a3dbf9bf81900be56c3" member_type: VOTER last_known_addr { host: "127.11.253.253" port: 35689 } } peers { permanent_uuid: "1d6dda3bf59b4d79a089e96bbd0b0774" member_type: VOTER last_known_addr { host: "127.11.253.252" port: 44101 } } }
I20260806 03:49:27.819566 31704 sys_catalog.cc:458] T 00000000000000000000000000000000 P e6f42db43a2b4a3dbf9bf81900be56c3 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 03:49:27.819689 31729 mvcc.cc:205] Tried to move back new op lower bound from 7315407535364038656 to 7315407535351427072. Current Snapshot: MvccSnapshot[applied={T|T < 7315407535364038656}]
I20260806 03:49:27.819959 31727 catalog_manager.cc:1347] Generated new cluster ID: 139f1a3b524f4f268d6c77953ea38c37
I20260806 03:49:27.820006 31727 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 03:49:27.825045 31725 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8400f5fc4fda4d069125f49070b47383 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "8400f5fc4fda4d069125f49070b47383" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8400f5fc4fda4d069125f49070b47383" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 42495 } } peers { permanent_uuid: "e6f42db43a2b4a3dbf9bf81900be56c3" member_type: VOTER last_known_addr { host: "127.11.253.253" port: 35689 } } peers { permanent_uuid: "1d6dda3bf59b4d79a089e96bbd0b0774" member_type: VOTER last_known_addr { host: "127.11.253.252" port: 44101 } } }
I20260806 03:49:27.825142 31725 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8400f5fc4fda4d069125f49070b47383 [sys.catalog]: This master's current role is: LEADER
I20260806 03:49:27.825378 31660 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1d6dda3bf59b4d79a089e96bbd0b0774 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "8400f5fc4fda4d069125f49070b47383" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8400f5fc4fda4d069125f49070b47383" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 42495 } } peers { permanent_uuid: "e6f42db43a2b4a3dbf9bf81900be56c3" member_type: VOTER last_known_addr { host: "127.11.253.253" port: 35689 } } peers { permanent_uuid: "1d6dda3bf59b4d79a089e96bbd0b0774" member_type: VOTER last_known_addr { host: "127.11.253.252" port: 44101 } } }
I20260806 03:49:27.825460 31660 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1d6dda3bf59b4d79a089e96bbd0b0774 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 03:49:27.849488 31727 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 03:49:27.850395 31727 catalog_manager.cc:1504] Loading token signing keys...
I20260806 03:49:27.855986 31727 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8400f5fc4fda4d069125f49070b47383: Generated new TSK 0
I20260806 03:49:27.856209 31727 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 03:49:28.622090 31674 catalog_manager.cc:1259] Loaded cluster ID: 139f1a3b524f4f268d6c77953ea38c37
I20260806 03:49:28.622169 31674 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 1d6dda3bf59b4d79a089e96bbd0b0774: loading cluster ID for follower catalog manager: success
I20260806 03:49:28.623036 31674 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 1d6dda3bf59b4d79a089e96bbd0b0774: acquiring CA information for follower catalog manager: success
I20260806 03:49:28.623368 31674 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 1d6dda3bf59b4d79a089e96bbd0b0774: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 03:49:28.678889 31723 catalog_manager.cc:1259] Loaded cluster ID: 139f1a3b524f4f268d6c77953ea38c37
I20260806 03:49:28.678972 31723 catalog_manager.cc:1552] T 00000000000000000000000000000000 P e6f42db43a2b4a3dbf9bf81900be56c3: loading cluster ID for follower catalog manager: success
I20260806 03:49:28.679865 31723 catalog_manager.cc:1574] T 00000000000000000000000000000000 P e6f42db43a2b4a3dbf9bf81900be56c3: acquiring CA information for follower catalog manager: success
I20260806 03:49:28.680200 31723 catalog_manager.cc:1602] T 00000000000000000000000000000000 P e6f42db43a2b4a3dbf9bf81900be56c3: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 03:49:28.919724 12279 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskToVWRe/build/release/bin/kudu with pid 31305
I20260806 03:49:28.930217 12279 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskToVWRe/build/release/bin/kudu with pid 31411
I20260806 03:49:28.939121 12279 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskToVWRe/build/release/bin/kudu with pid 31527
2026-08-06T03:49:28Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (3179 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785988004637995-12279-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 03:49:29 dist-test-slave-6ntk krb5kdc[31749](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 03:49:29 dist-test-slave-6ntk krb5kdc[31749](info): set up 2 sockets
Aug 06 03:49:29 dist-test-slave-6ntk krb5kdc[31749](info): commencing operation
krb5kdc: starting...
W20260806 03:49:30.215054 12279 mini_kdc.cc:121] Time spent starting KDC: real 1.250s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785988004637995-12279-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785988004637995-12279-0/krb5kdc/test-user.keytab.
Aug 06 03:49:30 dist-test-slave-6ntk krb5kdc[31749](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988170, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T03:49:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T03:49:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.253.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.253.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.253.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.254.keytab.
Entry for principal kudu/127.11.253.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.254.keytab.
I20260806 03:49:30.345844 12279 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskToVWRe/build/release/bin/kudu
/tmp/dist-test-taskToVWRe/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785988004637995-12279-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785988004637995-12279-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785988004637995-12279-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785988004637995-12279-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.253.254:41107
--webserver_interface=127.11.253.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.254.keytab
--principal=kudu/127.11.253.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.253.212:44545
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.253.254:41107
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785988004637995-12279-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785988004637995-12279-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785988004637995-12279-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:49:30.430231 31768 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:49:30.430524 31768 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:49:30.430562 31768 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:49:30.431748 31768 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 03:49:30.431826 31768 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:49:30.431860 31768 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:49:30.431888 31768 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 03:49:30.431921 31768 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 03:49:30.433408 31768 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.253.212:44545
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785988004637995-12279-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785988004637995-12279-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.253.254:41107
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.254.keytab
--principal=kudu/127.11.253.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.253.254:41107
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785988004637995-12279-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.253.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31768
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785988004637995-12279-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:49:30.433702 31768 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:49:30.434003 31768 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:49:30.439061 31782 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:49:30.439082 31778 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:49:30.439061 31777 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:49:30.439496 31768 server_base.cc:1034] running on GCE node
I20260806 03:49:30.439762 31768 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:49:30.440058 31768 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:49:30.441247 31768 hybrid_clock.cc:648] HybridClock initialized: now 1785988170441233 us; error 32 us; skew 500 ppm
Aug 06 03:49:30 dist-test-slave-6ntk krb5kdc[31749](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988170, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.253.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:49:30.443960 31768 init.cc:377] Logged in from keytab as kudu/127.11.253.254@KRBTEST.COM (short username kudu)
I20260806 03:49:30.444463 31768 webserver.cc:460] Webserver started at http://127.11.253.254:40883/ using document root <none> and password file <none>
I20260806 03:49:30.444695 31768 fs_manager.cc:359] Metadata directory not provided
I20260806 03:49:30.444739 31768 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:49:30.444914 31768 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:49:30.446112 31768 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785988004637995-12279-0/minicluster-data/master-0/data/instance:
uuid: "e5db7f168247438d860b1ccf8bfc48ac"
format_stamp: "Formatted at 2026-08-06 03:49:30 on dist-test-slave-6ntk"
server_key: "807469d7a3d04ffd5ee0e391108ab7b0"
server_key_iv: "0fecc99067e8cfde87f057868215fbd4"
server_key_version: "encryptionkey@0"
I20260806 03:49:30.446492 31768 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785988004637995-12279-0/minicluster-data/master-0/wal/instance:
uuid: "e5db7f168247438d860b1ccf8bfc48ac"
format_stamp: "Formatted at 2026-08-06 03:49:30 on dist-test-slave-6ntk"
server_key: "807469d7a3d04ffd5ee0e391108ab7b0"
server_key_iv: "0fecc99067e8cfde87f057868215fbd4"
server_key_version: "encryptionkey@0"
I20260806 03:49:30.447922 31768 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 03:49:30.448705 31797 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:49:30.448990 31768 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 03:49:30.449092 31768 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785988004637995-12279-0/minicluster-data/master-0/data,/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785988004637995-12279-0/minicluster-data/master-0/wal
uuid: "e5db7f168247438d860b1ccf8bfc48ac"
format_stamp: "Formatted at 2026-08-06 03:49:30 on dist-test-slave-6ntk"
server_key: "807469d7a3d04ffd5ee0e391108ab7b0"
server_key_iv: "0fecc99067e8cfde87f057868215fbd4"
server_key_version: "encryptionkey@0"
I20260806 03:49:30.449168 31768 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785988004637995-12279-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785988004637995-12279-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785988004637995-12279-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:49:30.474479 31768 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:49:30.476889 31768 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:49:30.477067 31768 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:49:30.481490 31768 rpc_server.cc:307] RPC server started. Bound to: 127.11.253.254:41107
I20260806 03:49:30.481724 31876 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.253.254:41107 every 8 connection(s)
I20260806 03:49:30.482009 31768 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785988004637995-12279-0/minicluster-data/master-0/data/info.pb
I20260806 03:49:30.482932 31877 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:49:30.485297 31877 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e5db7f168247438d860b1ccf8bfc48ac: Bootstrap starting.
I20260806 03:49:30.486027 31877 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e5db7f168247438d860b1ccf8bfc48ac: Neither blocks nor log segments found. Creating new log.
I20260806 03:49:30.486334 31877 log.cc:824] T 00000000000000000000000000000000 P e5db7f168247438d860b1ccf8bfc48ac: Log is configured to *not* fsync() on all Append() calls
I20260806 03:49:30.487170 31877 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e5db7f168247438d860b1ccf8bfc48ac: No bootstrap required, opened a new log
I20260806 03:49:30.488811 31877 raft_consensus.cc:348] T 00000000000000000000000000000000 P e5db7f168247438d860b1ccf8bfc48ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5db7f168247438d860b1ccf8bfc48ac" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 41107 } }
I20260806 03:49:30.488971 31877 raft_consensus.cc:374] T 00000000000000000000000000000000 P e5db7f168247438d860b1ccf8bfc48ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:49:30.488994 31877 raft_consensus.cc:729] T 00000000000000000000000000000000 P e5db7f168247438d860b1ccf8bfc48ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5db7f168247438d860b1ccf8bfc48ac, State: Initialized, Role: FOLLOWER
I20260806 03:49:30.489167 31877 consensus_queue.cc:260] T 00000000000000000000000000000000 P e5db7f168247438d860b1ccf8bfc48ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5db7f168247438d860b1ccf8bfc48ac" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 41107 } }
I20260806 03:49:30.489231 31877 raft_consensus.cc:388] T 00000000000000000000000000000000 P e5db7f168247438d860b1ccf8bfc48ac [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 03:49:30.489287 31877 raft_consensus.cc:482] T 00000000000000000000000000000000 P e5db7f168247438d860b1ccf8bfc48ac [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 03:49:30.489339 31877 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e5db7f168247438d860b1ccf8bfc48ac [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:49:30.489969 31877 raft_consensus.cc:504] T 00000000000000000000000000000000 P e5db7f168247438d860b1ccf8bfc48ac [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5db7f168247438d860b1ccf8bfc48ac" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 41107 } }
I20260806 03:49:30.490083 31877 leader_election.cc:302] T 00000000000000000000000000000000 P e5db7f168247438d860b1ccf8bfc48ac [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e5db7f168247438d860b1ccf8bfc48ac; no voters:
I20260806 03:49:30.490280 31877 leader_election.cc:288] T 00000000000000000000000000000000 P e5db7f168247438d860b1ccf8bfc48ac [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 03:49:30.490317 31882 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e5db7f168247438d860b1ccf8bfc48ac [term 1 FOLLOWER]: Leader election won for term 1
I20260806 03:49:30.490504 31882 raft_consensus.cc:686] T 00000000000000000000000000000000 P e5db7f168247438d860b1ccf8bfc48ac [term 1 LEADER]: Becoming Leader. State: Replica: e5db7f168247438d860b1ccf8bfc48ac, State: Running, Role: LEADER
I20260806 03:49:30.490556 31877 sys_catalog.cc:564] T 00000000000000000000000000000000 P e5db7f168247438d860b1ccf8bfc48ac [sys.catalog]: configured and running, proceeding with master startup.
I20260806 03:49:30.490643 31882 consensus_queue.cc:237] T 00000000000000000000000000000000 P e5db7f168247438d860b1ccf8bfc48ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5db7f168247438d860b1ccf8bfc48ac" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 41107 } }
I20260806 03:49:30.491250 31886 sys_catalog.cc:455] T 00000000000000000000000000000000 P e5db7f168247438d860b1ccf8bfc48ac [sys.catalog]: SysCatalogTable state changed. Reason: New leader e5db7f168247438d860b1ccf8bfc48ac. Latest consensus state: current_term: 1 leader_uuid: "e5db7f168247438d860b1ccf8bfc48ac" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5db7f168247438d860b1ccf8bfc48ac" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 41107 } } }
I20260806 03:49:30.491364 31886 sys_catalog.cc:458] T 00000000000000000000000000000000 P e5db7f168247438d860b1ccf8bfc48ac [sys.catalog]: This master's current role is: LEADER
I20260806 03:49:30.491531 31884 sys_catalog.cc:455] T 00000000000000000000000000000000 P e5db7f168247438d860b1ccf8bfc48ac [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e5db7f168247438d860b1ccf8bfc48ac" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5db7f168247438d860b1ccf8bfc48ac" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 41107 } } }
I20260806 03:49:30.491670 31884 sys_catalog.cc:458] T 00000000000000000000000000000000 P e5db7f168247438d860b1ccf8bfc48ac [sys.catalog]: This master's current role is: LEADER
I20260806 03:49:30.491694 31894 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 03:49:30.491891 12279 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskToVWRe/build/release/bin/kudu as pid 31768
I20260806 03:49:30.491974 12279 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785988004637995-12279-0/minicluster-data/master-0/wal/instance
I20260806 03:49:30.492197 12279 external_mini_cluster.cc:1428] Setting key aa5e43fd89fa65d774cac9bb3aa09d9a
I20260806 03:49:30.492254 31894 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 03:49:30.494104 31894 catalog_manager.cc:1347] Generated new cluster ID: f4b3ee7b4caf4db7998e8e8ae80ed0ba
I20260806 03:49:30.494156 31894 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 03:49:30 dist-test-slave-6ntk krb5kdc[31749](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988170, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.253.254@KRBTEST.COM
I20260806 03:49:30.500474 31894 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 03:49:30.501157 31894 catalog_manager.cc:1504] Loading token signing keys...
I20260806 03:49:30.503293 31894 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e5db7f168247438d860b1ccf8bfc48ac: Generated new TSK 0
I20260806 03:49:30.503468 31894 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 03:49:30.504745 31910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:49:30.492915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41250 (local address 127.11.253.254:41107)
0806 03:49:30.493090 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0806 03:49:30.493099 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 03:49:30.493466 (+ 367us) server_negotiation.cc:407] Connection header received
0806 03:49:30.493610 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:49:30.493615 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:49:30.493659 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:49:30.493826 (+ 167us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:49:30.494666 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:49:30.496089 (+ 1423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:49:30.497197 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:49:30.497778 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:49:30.497897 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:49:30.500982 (+ 3085us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:49:30.500986 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:49:30.500986 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:49:30.500998 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:49:30.503093 (+ 2095us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:49:30.503503 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:49:30.503506 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:49:30.503507 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:49:30.503611 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:49:30.503817 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:49:30.503818 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:49:30.503818 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:49:30.504124 (+ 306us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:49:30.504277 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0806 03:49:30.504348 (+ 71us) server_negotiation.cc:299] Negotiation successful
0806 03:49:30.504502 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":83,"threads_started":1}
I20260806 03:49:30.505892 12279 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 03:49:30.508960 12279 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskToVWRe/build/release/bin/kudu with pid 31768
2026-08-06T03:49:30Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1569 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 03:49:30 dist-test-slave-6ntk krb5kdc[31917](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 03:49:30 dist-test-slave-6ntk krb5kdc[31917](info): set up 2 sockets
Aug 06 03:49:30 dist-test-slave-6ntk krb5kdc[31917](info): commencing operation
krb5kdc: starting...
W20260806 03:49:31.712466 12279 mini_kdc.cc:121] Time spent starting KDC: real 1.178s user 0.005s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/krb5kdc/test-user.keytab.
Aug 06 03:49:31 dist-test-slave-6ntk krb5kdc[31917](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988171, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T03:49:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T03:49:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.253.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.253.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.253.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.254.keytab.
Entry for principal kudu/127.11.253.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.254.keytab.
I20260806 03:49:31.842833 12279 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskToVWRe/build/release/bin/kudu
/tmp/dist-test-taskToVWRe/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.253.254:44939
--webserver_interface=127.11.253.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.254.keytab
--principal=kudu/127.11.253.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.253.212:40573
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.253.254:44939
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:49:31.929309 31937 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:49:31.929589 31937 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:49:31.929627 31937 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:49:31.930830 31937 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 03:49:31.930907 31937 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:49:31.930930 31937 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:49:31.930953 31937 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 03:49:31.930982 31937 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 03:49:31.932560 31937 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.253.212:40573
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.253.254:44939
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.254.keytab
--principal=kudu/127.11.253.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.253.254:44939
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.253.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31937
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:49:31.932869 31937 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:49:31.933142 31937 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:49:31.938395 31942 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:49:31.938426 31943 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:49:31.938485 31947 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:49:31.938784 31937 server_base.cc:1034] running on GCE node
I20260806 03:49:31.939096 31937 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:49:31.939407 31937 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:49:31.940589 31937 hybrid_clock.cc:648] HybridClock initialized: now 1785988171940577 us; error 49 us; skew 500 ppm
Aug 06 03:49:31 dist-test-slave-6ntk krb5kdc[31917](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988171, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.253.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:49:31.947574 31937 init.cc:377] Logged in from keytab as kudu/127.11.253.254@KRBTEST.COM (short username kudu)
I20260806 03:49:31.948114 31937 webserver.cc:460] Webserver started at http://127.11.253.254:38907/ using document root <none> and password file <none>
I20260806 03:49:31.948364 31937 fs_manager.cc:359] Metadata directory not provided
I20260806 03:49:31.948436 31937 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:49:31.948596 31937 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:49:31.949733 31937 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/minicluster-data/master-0/data/instance:
uuid: "9c6e110e6ae045d2992beccbc2b49594"
format_stamp: "Formatted at 2026-08-06 03:49:31 on dist-test-slave-6ntk"
server_key: "c1b753a64abc5121b494e9ad05f3ce2c"
server_key_iv: "dbed1853eafb71a198456bd471eec18d"
server_key_version: "encryptionkey@0"
I20260806 03:49:31.950169 31937 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/minicluster-data/master-0/wal/instance:
uuid: "9c6e110e6ae045d2992beccbc2b49594"
format_stamp: "Formatted at 2026-08-06 03:49:31 on dist-test-slave-6ntk"
server_key: "c1b753a64abc5121b494e9ad05f3ce2c"
server_key_iv: "dbed1853eafb71a198456bd471eec18d"
server_key_version: "encryptionkey@0"
I20260806 03:49:31.951598 31937 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 03:49:31.952404 31958 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:49:31.952591 31937 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 03:49:31.952704 31937 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/minicluster-data/master-0/data,/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/minicluster-data/master-0/wal
uuid: "9c6e110e6ae045d2992beccbc2b49594"
format_stamp: "Formatted at 2026-08-06 03:49:31 on dist-test-slave-6ntk"
server_key: "c1b753a64abc5121b494e9ad05f3ce2c"
server_key_iv: "dbed1853eafb71a198456bd471eec18d"
server_key_version: "encryptionkey@0"
I20260806 03:49:31.952795 31937 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:49:31.975389 31937 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:49:31.977447 31937 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:49:31.977624 31937 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:49:31.981771 31937 rpc_server.cc:307] RPC server started. Bound to: 127.11.253.254:44939
I20260806 03:49:31.981837 32039 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.253.254:44939 every 8 connection(s)
I20260806 03:49:31.982244 31937 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/minicluster-data/master-0/data/info.pb
I20260806 03:49:31.983243 32040 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:49:31.985800 32040 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9c6e110e6ae045d2992beccbc2b49594: Bootstrap starting.
I20260806 03:49:31.986457 32040 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9c6e110e6ae045d2992beccbc2b49594: Neither blocks nor log segments found. Creating new log.
I20260806 03:49:31.986696 32040 log.cc:824] T 00000000000000000000000000000000 P 9c6e110e6ae045d2992beccbc2b49594: Log is configured to *not* fsync() on all Append() calls
I20260806 03:49:31.987406 32040 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9c6e110e6ae045d2992beccbc2b49594: No bootstrap required, opened a new log
I20260806 03:49:31.988879 12279 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskToVWRe/build/release/bin/kudu as pid 31937
I20260806 03:49:31.989001 12279 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/minicluster-data/master-0/wal/instance
I20260806 03:49:31.989204 12279 external_mini_cluster.cc:1428] Setting key eb9d798c60967b0b9ebec3872fd9e406
I20260806 03:49:31.989277 32040 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9c6e110e6ae045d2992beccbc2b49594 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c6e110e6ae045d2992beccbc2b49594" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 44939 } }
I20260806 03:49:31.989464 32040 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9c6e110e6ae045d2992beccbc2b49594 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:49:31.989493 32040 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9c6e110e6ae045d2992beccbc2b49594 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c6e110e6ae045d2992beccbc2b49594, State: Initialized, Role: FOLLOWER
I20260806 03:49:31.989670 32040 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9c6e110e6ae045d2992beccbc2b49594 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c6e110e6ae045d2992beccbc2b49594" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 44939 } }
I20260806 03:49:31.989792 32040 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9c6e110e6ae045d2992beccbc2b49594 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 03:49:31.989856 32040 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9c6e110e6ae045d2992beccbc2b49594 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 03:49:31.989939 32040 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9c6e110e6ae045d2992beccbc2b49594 [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:49:31.991170 32040 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9c6e110e6ae045d2992beccbc2b49594 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c6e110e6ae045d2992beccbc2b49594" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 44939 } }
I20260806 03:49:31.991289 32040 leader_election.cc:302] T 00000000000000000000000000000000 P 9c6e110e6ae045d2992beccbc2b49594 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9c6e110e6ae045d2992beccbc2b49594; no voters:
I20260806 03:49:31.991449 32040 leader_election.cc:288] T 00000000000000000000000000000000 P 9c6e110e6ae045d2992beccbc2b49594 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 03:49:31.991552 32045 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9c6e110e6ae045d2992beccbc2b49594 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 03:49:31.991741 32040 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9c6e110e6ae045d2992beccbc2b49594 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 03:49:31.991755 32045 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9c6e110e6ae045d2992beccbc2b49594 [term 1 LEADER]: Becoming Leader. State: Replica: 9c6e110e6ae045d2992beccbc2b49594, State: Running, Role: LEADER
I20260806 03:49:31.991971 32045 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9c6e110e6ae045d2992beccbc2b49594 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c6e110e6ae045d2992beccbc2b49594" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 44939 } }
I20260806 03:49:31.992535 32048 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9c6e110e6ae045d2992beccbc2b49594 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9c6e110e6ae045d2992beccbc2b49594" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c6e110e6ae045d2992beccbc2b49594" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 44939 } } }
I20260806 03:49:31.992645 32048 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9c6e110e6ae045d2992beccbc2b49594 [sys.catalog]: This master's current role is: LEADER
I20260806 03:49:31.992527 32045 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9c6e110e6ae045d2992beccbc2b49594 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9c6e110e6ae045d2992beccbc2b49594. Latest consensus state: current_term: 1 leader_uuid: "9c6e110e6ae045d2992beccbc2b49594" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c6e110e6ae045d2992beccbc2b49594" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 44939 } } }
W20260806 03:49:31.993583 32067 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9c6e110e6ae045d2992beccbc2b49594: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260806 03:49:31.994065 32045 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9c6e110e6ae045d2992beccbc2b49594 [sys.catalog]: This master's current role is: LEADER
W20260806 03:49:31.994076 32067 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 03:49:31.994494 32060 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 03:49:31.994699 32060 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 03:49:31.996485 32060 catalog_manager.cc:1347] Generated new cluster ID: b5c142d5173b48c8b1eaa358a27de72b
Aug 06 03:49:31 dist-test-slave-6ntk krb5kdc[31917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988171, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.253.254@KRBTEST.COM
I20260806 03:49:31.996608 32060 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 03:49:32.000546 32044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:49:31.989800 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36788 (local address 127.11.253.254:44939)
0806 03:49:31.990052 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0806 03:49:31.990063 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 03:49:31.990118 (+ 55us) server_negotiation.cc:407] Connection header received
0806 03:49:31.990252 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:49:31.990259 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:49:31.990316 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:49:31.990484 (+ 168us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:49:31.991111 (+ 627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:49:31.992549 (+ 1438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:49:31.993685 (+ 1136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:49:31.994243 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:49:31.994377 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:49:31.997127 (+ 2750us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:49:31.997138 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:49:31.997138 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:49:31.997150 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:49:31.998966 (+ 1816us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:49:31.999336 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:49:31.999340 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:49:31.999340 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:49:31.999448 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:49:31.999673 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:49:31.999673 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:49:31.999674 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:49:31.999925 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:49:32.000105 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0806 03:49:32.000211 (+ 106us) server_negotiation.cc:299] Negotiation successful
0806 03:49:32.000362 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":166,"thread_start_us":129,"threads_started":1}
I20260806 03:49:32.014532 32060 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 03:49:32.015317 32060 catalog_manager.cc:1504] Loading token signing keys...
I20260806 03:49:32.024004 32060 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9c6e110e6ae045d2992beccbc2b49594: Generated new TSK 0
I20260806 03:49:32.024255 32060 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.253.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.253.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.253.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.193.keytab.
Entry for principal kudu/127.11.253.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.193.keytab.
I20260806 03:49:32.069329 12279 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskToVWRe/build/release/bin/kudu
/tmp/dist-test-taskToVWRe/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.253.193:0
--local_ip_for_outbound_sockets=127.11.253.193
--webserver_interface=127.11.253.193
--webserver_port=0
--tserver_master_addrs=127.11.253.254:44939
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.193.keytab
--principal=kudu/127.11.253.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.253.212:40573
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:49:32.194353 32076 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:49:32.194726 32076 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:49:32.194810 32076 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:49:32.196532 32076 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:49:32.196591 32076 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:49:32.196623 32076 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.253.193
I20260806 03:49:32.198316 32076 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.253.212:40573
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.193.keytab
--principal=kudu/127.11.253.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.253.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.253.193
--webserver_port=0
--tserver_master_addrs=127.11.253.254:44939
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32076
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.253.193
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:49:32.198604 32076 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:49:32.198945 32076 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:49:32.205160 32082 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:49:32.205214 32081 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:49:32.205397 32085 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:49:32.205544 32076 server_base.cc:1034] running on GCE node
I20260806 03:49:32.205807 32076 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:49:32.206157 32076 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:49:32.207365 32076 hybrid_clock.cc:648] HybridClock initialized: now 1785988172207353 us; error 32 us; skew 500 ppm
Aug 06 03:49:32 dist-test-slave-6ntk krb5kdc[31917](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988172, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.253.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:49:32.216439 32076 init.cc:377] Logged in from keytab as kudu/127.11.253.193@KRBTEST.COM (short username kudu)
I20260806 03:49:32.216984 32076 webserver.cc:460] Webserver started at http://127.11.253.193:36067/ using document root <none> and password file <none>
I20260806 03:49:32.217239 32076 fs_manager.cc:359] Metadata directory not provided
I20260806 03:49:32.217291 32076 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:49:32.217433 32076 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:49:32.218645 32076 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/minicluster-data/ts-0/data/instance:
uuid: "67abaec1b40248afb1a33088431ae1d5"
format_stamp: "Formatted at 2026-08-06 03:49:32 on dist-test-slave-6ntk"
server_key: "97e4da2295ff60317613732a64c0f93b"
server_key_iv: "94348828464dd14f650c05dac0d9e775"
server_key_version: "encryptionkey@0"
I20260806 03:49:32.219034 32076 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/minicluster-data/ts-0/wal/instance:
uuid: "67abaec1b40248afb1a33088431ae1d5"
format_stamp: "Formatted at 2026-08-06 03:49:32 on dist-test-slave-6ntk"
server_key: "97e4da2295ff60317613732a64c0f93b"
server_key_iv: "94348828464dd14f650c05dac0d9e775"
server_key_version: "encryptionkey@0"
I20260806 03:49:32.220477 32076 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 03:49:32.221263 32096 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:49:32.221450 32076 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 03:49:32.221567 32076 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/minicluster-data/ts-0/data,/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/minicluster-data/ts-0/wal
uuid: "67abaec1b40248afb1a33088431ae1d5"
format_stamp: "Formatted at 2026-08-06 03:49:32 on dist-test-slave-6ntk"
server_key: "97e4da2295ff60317613732a64c0f93b"
server_key_iv: "94348828464dd14f650c05dac0d9e775"
server_key_version: "encryptionkey@0"
I20260806 03:49:32.221643 32076 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:49:32.234491 32076 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:49:32.236480 32076 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:49:32.236668 32076 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:49:32.236931 32076 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 03:49:32.237272 32076 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 03:49:32.237329 32076 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 03:49:32.237365 32076 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 03:49:32.237394 32076 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 03:49:32.243490 32076 rpc_server.cc:307] RPC server started. Bound to: 127.11.253.193:36851
I20260806 03:49:32.243547 32257 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.253.193:36851 every 8 connection(s)
I20260806 03:49:32.243880 32076 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/minicluster-data/ts-0/data/info.pb
I20260806 03:49:32.246912 12279 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskToVWRe/build/release/bin/kudu as pid 32076
I20260806 03:49:32.247027 12279 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/minicluster-data/ts-0/wal/instance
I20260806 03:49:32.247233 12279 external_mini_cluster.cc:1428] Setting key bdcef008bfd54a1b5c3959004eead311
Aug 06 03:49:32 dist-test-slave-6ntk krb5kdc[31917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988172, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.253.193@KRBTEST.COM for kudu/127.11.253.254@KRBTEST.COM
I20260806 03:49:32.256282 32044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:49:32.244670 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.253.193:43113 (local address 127.11.253.254:44939)
0806 03:49:32.244719 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0806 03:49:32.244721 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:49:32.244968 (+ 247us) server_negotiation.cc:407] Connection header received
0806 03:49:32.245530 (+ 562us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:49:32.245532 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:49:32.245550 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:49:32.245602 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:49:32.246650 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:49:32.247473 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:49:32.248768 (+ 1295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:49:32.249090 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:49:32.249169 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:49:32.253127 (+ 3958us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:49:32.253129 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:49:32.253129 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:49:32.253138 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:49:32.254698 (+ 1560us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:49:32.255202 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:49:32.255203 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:49:32.255204 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:49:32.255269 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:49:32.255553 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:49:32.255554 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:49:32.255554 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:49:32.255678 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:49:32.255759 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0806 03:49:32.256055 (+ 296us) server_negotiation.cc:299] Negotiation successful
0806 03:49:32.256183 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260806 03:49:32.256488 32260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:49:32.244630 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:44939 (local address 127.11.253.193:43113)
0806 03:49:32.244821 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0806 03:49:32.244865 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0806 03:49:32.245416 (+ 551us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:49:32.245675 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:49:32.245679 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:49:32.246331 (+ 652us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:49:32.246535 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:49:32.246539 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:49:32.247621 (+ 1082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:49:32.247622 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:49:32.248662 (+ 1040us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:49:32.248666 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:49:32.249186 (+ 520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:49:32.249187 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:49:32.249322 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:49:32.251264 (+ 1942us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:49:32.251273 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:49:32.253023 (+ 1750us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:49:32.254808 (+ 1785us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:49:32.254817 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:49:32.254819 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:49:32.255107 (+ 288us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:49:32.255339 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:49:32.255340 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:49:32.255340 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:49:32.255493 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:49:32.255749 (+ 256us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:49:32.255757 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:49:32.255914 (+ 157us) client_negotiation.cc:769] Sending connection context
0806 03:49:32.256031 (+ 117us) client_negotiation.cc:240] Negotiation successful
0806 03:49:32.256207 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":88,"threads_started":1}
WARNING: no policy specified for kudu/127.11.253.194@KRBTEST.COM; defaulting to no policy
I20260806 03:49:32.257253 32258 heartbeater.cc:347] Connected to a master server at 127.11.253.254:44939
I20260806 03:49:32.257385 32258 heartbeater.cc:464] Registering TS with master...
I20260806 03:49:32.257582 32258 heartbeater.cc:511] Master 127.11.253.254:44939 requested a full tablet report, sending...
I20260806 03:49:32.258313 31983 ts_manager.cc:194] Registered new tserver with Master: 67abaec1b40248afb1a33088431ae1d5 (127.11.253.193:36851)
I20260806 03:49:32.258967 31983 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.253.193@KRBTEST.COM'} at 127.11.253.193:43113
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.253.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.253.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.194.keytab.
Entry for principal kudu/127.11.253.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.194.keytab.
I20260806 03:49:32.278167 12279 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskToVWRe/build/release/bin/kudu
/tmp/dist-test-taskToVWRe/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.253.194:0
--local_ip_for_outbound_sockets=127.11.253.194
--webserver_interface=127.11.253.194
--webserver_port=0
--tserver_master_addrs=127.11.253.254:44939
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.194.keytab
--principal=kudu/127.11.253.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.253.212:40573
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:49:32.364043 32266 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:49:32.364319 32266 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:49:32.364357 32266 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:49:32.365533 32266 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:49:32.365604 32266 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:49:32.365648 32266 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.253.194
I20260806 03:49:32.367175 32266 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.253.212:40573
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.194.keytab
--principal=kudu/127.11.253.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.253.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.253.194
--webserver_port=0
--tserver_master_addrs=127.11.253.254:44939
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32266
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.253.194
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:49:32.367473 32266 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:49:32.367720 32266 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:49:32.373045 32271 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:49:32.373152 32275 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:49:32.373205 32266 server_base.cc:1034] running on GCE node
W20260806 03:49:32.373165 32272 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:49:32.373732 32266 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:49:32.374066 32266 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:49:32.375254 32266 hybrid_clock.cc:648] HybridClock initialized: now 1785988172375210 us; error 55 us; skew 500 ppm
Aug 06 03:49:32 dist-test-slave-6ntk krb5kdc[31917](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988172, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.253.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:49:32.378046 32266 init.cc:377] Logged in from keytab as kudu/127.11.253.194@KRBTEST.COM (short username kudu)
I20260806 03:49:32.378548 32266 webserver.cc:460] Webserver started at http://127.11.253.194:39759/ using document root <none> and password file <none>
I20260806 03:49:32.378819 32266 fs_manager.cc:359] Metadata directory not provided
I20260806 03:49:32.378866 32266 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:49:32.379030 32266 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:49:32.380620 32266 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/minicluster-data/ts-1/data/instance:
uuid: "df657246a4be41cdb398d7c7e988d8e4"
format_stamp: "Formatted at 2026-08-06 03:49:32 on dist-test-slave-6ntk"
server_key: "b713a4254580841280323bb428a9e14f"
server_key_iv: "16bdd544f6ea35651e971f1ea83e3d4e"
server_key_version: "encryptionkey@0"
I20260806 03:49:32.381024 32266 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/minicluster-data/ts-1/wal/instance:
uuid: "df657246a4be41cdb398d7c7e988d8e4"
format_stamp: "Formatted at 2026-08-06 03:49:32 on dist-test-slave-6ntk"
server_key: "b713a4254580841280323bb428a9e14f"
server_key_iv: "16bdd544f6ea35651e971f1ea83e3d4e"
server_key_version: "encryptionkey@0"
I20260806 03:49:32.382484 32266 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.004s sys 0.000s
I20260806 03:49:32.383288 32288 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:49:32.383455 32266 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 03:49:32.383560 32266 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/minicluster-data/ts-1/data,/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/minicluster-data/ts-1/wal
uuid: "df657246a4be41cdb398d7c7e988d8e4"
format_stamp: "Formatted at 2026-08-06 03:49:32 on dist-test-slave-6ntk"
server_key: "b713a4254580841280323bb428a9e14f"
server_key_iv: "16bdd544f6ea35651e971f1ea83e3d4e"
server_key_version: "encryptionkey@0"
I20260806 03:49:32.383639 32266 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:49:32.405383 32266 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:49:32.407931 32266 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:49:32.408048 32266 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:49:32.408259 32266 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 03:49:32.408525 32266 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 03:49:32.408556 32266 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 03:49:32.408579 32266 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 03:49:32.408591 32266 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 03:49:32.414546 32266 rpc_server.cc:307] RPC server started. Bound to: 127.11.253.194:37941
I20260806 03:49:32.414646 32449 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.253.194:37941 every 8 connection(s)
I20260806 03:49:32.414974 32266 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/minicluster-data/ts-1/data/info.pb
Aug 06 03:49:32 dist-test-slave-6ntk krb5kdc[31917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988172, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.253.194@KRBTEST.COM for kudu/127.11.253.254@KRBTEST.COM
I20260806 03:49:32.424249 12279 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskToVWRe/build/release/bin/kudu as pid 32266
I20260806 03:49:32.424397 12279 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/minicluster-data/ts-1/wal/instance
I20260806 03:49:32.424665 12279 external_mini_cluster.cc:1428] Setting key 9d398e0f6faaae38aa18119e0283cb65
I20260806 03:49:32.424913 32044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:49:32.415761 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.253.194:57245 (local address 127.11.253.254:44939)
0806 03:49:32.415808 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0806 03:49:32.415810 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:49:32.416017 (+ 207us) server_negotiation.cc:407] Connection header received
0806 03:49:32.416572 (+ 555us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:49:32.416573 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:49:32.416583 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:49:32.416642 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:49:32.417569 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:49:32.418110 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:49:32.419222 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:49:32.419418 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:49:32.419470 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:49:32.421754 (+ 2284us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:49:32.421756 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:49:32.421757 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:49:32.421765 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:49:32.423295 (+ 1530us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:49:32.423739 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:49:32.423740 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:49:32.423741 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:49:32.423807 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:49:32.424101 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:49:32.424102 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:49:32.424103 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:49:32.424231 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:49:32.424314 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0806 03:49:32.424688 (+ 374us) server_negotiation.cc:299] Negotiation successful
0806 03:49:32.424808 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260806 03:49:32.425077 32456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:49:32.415698 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:44939 (local address 127.11.253.194:57245)
0806 03:49:32.415897 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0806 03:49:32.415940 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 03:49:32.416481 (+ 541us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:49:32.416699 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:49:32.416702 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:49:32.417291 (+ 589us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:49:32.417487 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:49:32.417491 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:49:32.418225 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:49:32.418227 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:49:32.419133 (+ 906us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:49:32.419136 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:49:32.419498 (+ 362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:49:32.419498 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:49:32.419612 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:49:32.420113 (+ 501us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:49:32.420122 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:49:32.421642 (+ 1520us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:49:32.423407 (+ 1765us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:49:32.423415 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:49:32.423417 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:49:32.423652 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:49:32.423894 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:49:32.423895 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:49:32.423896 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:49:32.424014 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:49:32.424304 (+ 290us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:49:32.424311 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:49:32.424515 (+ 204us) client_negotiation.cc:769] Sending connection context
0806 03:49:32.424648 (+ 133us) client_negotiation.cc:240] Negotiation successful
0806 03:49:32.424832 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":106,"threads_started":1}
I20260806 03:49:32.425760 32450 heartbeater.cc:347] Connected to a master server at 127.11.253.254:44939
I20260806 03:49:32.425848 32450 heartbeater.cc:464] Registering TS with master...
I20260806 03:49:32.426095 32450 heartbeater.cc:511] Master 127.11.253.254:44939 requested a full tablet report, sending...
I20260806 03:49:32.426530 31983 ts_manager.cc:194] Registered new tserver with Master: df657246a4be41cdb398d7c7e988d8e4 (127.11.253.194:37941)
I20260806 03:49:32.426927 31983 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.253.194@KRBTEST.COM'} at 127.11.253.194:57245
WARNING: no policy specified for kudu/127.11.253.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.253.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.253.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.195.keytab.
Entry for principal kudu/127.11.253.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.195.keytab.
I20260806 03:49:32.452749 12279 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskToVWRe/build/release/bin/kudu
/tmp/dist-test-taskToVWRe/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.253.195:0
--local_ip_for_outbound_sockets=127.11.253.195
--webserver_interface=127.11.253.195
--webserver_port=0
--tserver_master_addrs=127.11.253.254:44939
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.195.keytab
--principal=kudu/127.11.253.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.253.212:40573
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:49:32.538163 32459 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:49:32.538371 32459 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:49:32.538395 32459 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:49:32.539546 32459 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:49:32.539592 32459 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:49:32.539618 32459 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.253.195
I20260806 03:49:32.541071 32459 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.253.212:40573
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.195.keytab
--principal=kudu/127.11.253.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.253.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.253.195
--webserver_port=0
--tserver_master_addrs=127.11.253.254:44939
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32459
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.253.195
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:49:32.541325 32459 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:49:32.541561 32459 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:49:32.546800 32469 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:49:32.546823 32459 server_base.cc:1034] running on GCE node
W20260806 03:49:32.546810 32467 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:49:32.546864 32466 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:49:32.547176 32459 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:49:32.547434 32459 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:49:32.548590 32459 hybrid_clock.cc:648] HybridClock initialized: now 1785988172548572 us; error 47 us; skew 500 ppm
Aug 06 03:49:32 dist-test-slave-6ntk krb5kdc[31917](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988172, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.253.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:49:32.551411 32459 init.cc:377] Logged in from keytab as kudu/127.11.253.195@KRBTEST.COM (short username kudu)
I20260806 03:49:32.551807 32459 webserver.cc:460] Webserver started at http://127.11.253.195:43749/ using document root <none> and password file <none>
I20260806 03:49:32.552052 32459 fs_manager.cc:359] Metadata directory not provided
I20260806 03:49:32.552122 32459 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:49:32.552279 32459 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:49:32.553341 32459 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/minicluster-data/ts-2/data/instance:
uuid: "403aacadd2b546c1b7289fad1f6066eb"
format_stamp: "Formatted at 2026-08-06 03:49:32 on dist-test-slave-6ntk"
server_key: "4d1b136f6886e0b91ec96dacee8cc8da"
server_key_iv: "66f109e8c54a9e4fbe7df92edd2efe14"
server_key_version: "encryptionkey@0"
I20260806 03:49:32.553719 32459 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/minicluster-data/ts-2/wal/instance:
uuid: "403aacadd2b546c1b7289fad1f6066eb"
format_stamp: "Formatted at 2026-08-06 03:49:32 on dist-test-slave-6ntk"
server_key: "4d1b136f6886e0b91ec96dacee8cc8da"
server_key_iv: "66f109e8c54a9e4fbe7df92edd2efe14"
server_key_version: "encryptionkey@0"
I20260806 03:49:32.555109 32459 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 03:49:32.556069 32481 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:49:32.556295 32459 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 03:49:32.556391 32459 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/minicluster-data/ts-2/data,/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/minicluster-data/ts-2/wal
uuid: "403aacadd2b546c1b7289fad1f6066eb"
format_stamp: "Formatted at 2026-08-06 03:49:32 on dist-test-slave-6ntk"
server_key: "4d1b136f6886e0b91ec96dacee8cc8da"
server_key_iv: "66f109e8c54a9e4fbe7df92edd2efe14"
server_key_version: "encryptionkey@0"
I20260806 03:49:32.556444 32459 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:49:32.566919 32459 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:49:32.568871 32459 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:49:32.568980 32459 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:49:32.569154 32459 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 03:49:32.569470 32459 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 03:49:32.569528 32459 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 03:49:32.569567 32459 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 03:49:32.569594 32459 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 03:49:32.575565 32459 rpc_server.cc:307] RPC server started. Bound to: 127.11.253.195:39103
I20260806 03:49:32.575655 32655 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.253.195:39103 every 8 connection(s)
I20260806 03:49:32.575999 32459 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/minicluster-data/ts-2/data/info.pb
I20260806 03:49:32.578068 12279 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskToVWRe/build/release/bin/kudu as pid 32459
I20260806 03:49:32.578145 12279 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/minicluster-data/ts-2/wal/instance
I20260806 03:49:32.578300 12279 external_mini_cluster.cc:1428] Setting key 6731394542acca9334e34786c4a6e2f0
Aug 06 03:49:32 dist-test-slave-6ntk krb5kdc[31917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988172, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.253.195@KRBTEST.COM for kudu/127.11.253.254@KRBTEST.COM
I20260806 03:49:32.585597 32044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:49:32.576796 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.253.195:42609 (local address 127.11.253.254:44939)
0806 03:49:32.576847 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0806 03:49:32.576849 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:49:32.577062 (+ 213us) server_negotiation.cc:407] Connection header received
0806 03:49:32.577620 (+ 558us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:49:32.577622 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:49:32.577635 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:49:32.577694 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:49:32.579023 (+ 1329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:49:32.579657 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:49:32.580632 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:49:32.580818 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:49:32.580873 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:49:32.583193 (+ 2320us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:49:32.583194 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:49:32.583194 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:49:32.583201 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:49:32.584448 (+ 1247us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:49:32.584809 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:49:32.584810 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:49:32.584810 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:49:32.584864 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:49:32.585077 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:49:32.585077 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:49:32.585078 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:49:32.585176 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:49:32.585225 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0806 03:49:32.585471 (+ 246us) server_negotiation.cc:299] Negotiation successful
0806 03:49:32.585532 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260806 03:49:32.585800 32658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:49:32.576751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:44939 (local address 127.11.253.195:42609)
0806 03:49:32.576918 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0806 03:49:32.576964 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0806 03:49:32.577507 (+ 543us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:49:32.577763 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:49:32.577767 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:49:32.578605 (+ 838us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:49:32.578928 (+ 323us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:49:32.578935 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:49:32.579739 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:49:32.579740 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:49:32.580536 (+ 796us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:49:32.580538 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:49:32.580922 (+ 384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:49:32.580923 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:49:32.581046 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:49:32.581647 (+ 601us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:49:32.581656 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:49:32.583113 (+ 1457us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:49:32.584519 (+ 1406us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:49:32.584526 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:49:32.584527 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:49:32.584746 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:49:32.584922 (+ 176us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:49:32.584922 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:49:32.584923 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:49:32.585022 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:49:32.585234 (+ 212us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:49:32.585239 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:49:32.585356 (+ 117us) client_negotiation.cc:769] Sending connection context
0806 03:49:32.585443 (+ 87us) client_negotiation.cc:240] Negotiation successful
0806 03:49:32.585564 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":77,"threads_started":1}
I20260806 03:49:32.586330 32656 heartbeater.cc:347] Connected to a master server at 127.11.253.254:44939
I20260806 03:49:32.586447 32656 heartbeater.cc:464] Registering TS with master...
I20260806 03:49:32.586634 32656 heartbeater.cc:511] Master 127.11.253.254:44939 requested a full tablet report, sending...
I20260806 03:49:32.587025 31983 ts_manager.cc:194] Registered new tserver with Master: 403aacadd2b546c1b7289fad1f6066eb (127.11.253.195:39103)
I20260806 03:49:32.587430 31983 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.253.195@KRBTEST.COM'} at 127.11.253.195:42609
I20260806 03:49:32.589836 12279 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 03:49:32.597283 32044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:49:32.591213 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36800 (local address 127.11.253.254:44939)
0806 03:49:32.591268 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0806 03:49:32.591270 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:49:32.591441 (+ 171us) server_negotiation.cc:407] Connection header received
0806 03:49:32.591499 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:49:32.591499 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:49:32.591513 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:49:32.591561 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:49:32.592332 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:49:32.592899 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:49:32.593650 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:49:32.593841 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:49:32.593969 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:49:32.595157 (+ 1188us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:49:32.595158 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:49:32.595158 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:49:32.595163 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:49:32.596306 (+ 1143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:49:32.596649 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:49:32.596650 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:49:32.596650 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:49:32.596699 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:49:32.596868 (+ 169us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:49:32.596868 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:49:32.596869 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:49:32.597002 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:49:32.597079 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0806 03:49:32.597167 (+ 88us) server_negotiation.cc:299] Negotiation successful
0806 03:49:32.597223 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260806 03:49:32.598851 31983 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36800:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 03:49:32.600476 31983 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 03:49:32.607206 32678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:49:32.602914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.193:36851 (local address 127.0.0.1:48746)
0806 03:49:32.603213 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0806 03:49:32.603245 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0806 03:49:32.603312 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:49:32.603716 (+ 404us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:49:32.603719 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:49:32.603722 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 03:49:32.603811 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:49:32.603813 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:49:32.604827 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:49:32.604828 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:49:32.606082 (+ 1254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:49:32.606085 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:49:32.606879 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:49:32.606880 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:49:32.606961 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:49:32.606974 (+ 13us) client_negotiation.cc:769] Sending connection context
0806 03:49:32.607066 (+ 92us) client_negotiation.cc:240] Negotiation successful
0806 03:49:32.607122 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":286,"mutex_wait_us":2,"thread_start_us":98,"threads_started":1}
I20260806 03:49:32.607211 32679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:49:32.603177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48746 (local address 127.11.253.193:36851)
0806 03:49:32.603525 (+ 348us) server_negotiation.cc:206] Beginning negotiation
0806 03:49:32.603528 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 03:49:32.603536 (+ 8us) server_negotiation.cc:407] Connection header received
0806 03:49:32.603570 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:49:32.603571 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:49:32.603605 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:49:32.603695 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 03:49:32.603915 (+ 220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:49:32.604720 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:49:32.606169 (+ 1449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:49:32.606771 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:49:32.606875 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:49:32.606938 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0806 03:49:32.607083 (+ 145us) server_negotiation.cc:299] Negotiation successful
0806 03:49:32.607127 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":319,"thread_start_us":85,"threads_started":1}
I20260806 03:49:32.607468 32670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:49:32.602873 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.195:39103 (local address 127.0.0.1:39998)
0806 03:49:32.603191 (+ 318us) negotiation.cc:107] Waiting for socket to connect
0806 03:49:32.603245 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0806 03:49:32.603312 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:49:32.603720 (+ 408us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:49:32.603721 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:49:32.603722 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 03:49:32.603835 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:49:32.603838 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:49:32.605162 (+ 1324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:49:32.605163 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:49:32.606243 (+ 1080us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:49:32.606245 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:49:32.607227 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:49:32.607228 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:49:32.607294 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:49:32.607298 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 03:49:32.607359 (+ 61us) client_negotiation.cc:240] Negotiation successful
0806 03:49:32.607398 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":289,"mutex_wait_us":41,"thread_start_us":75,"threads_started":1}
I20260806 03:49:32.607686 32671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:49:32.603102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.194:37941 (local address 127.0.0.1:51988)
0806 03:49:32.603776 (+ 674us) negotiation.cc:107] Waiting for socket to connect
0806 03:49:32.603787 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 03:49:32.603877 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:49:32.604081 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:49:32.604082 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:49:32.604084 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 03:49:32.604120 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:49:32.604121 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:49:32.605297 (+ 1176us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:49:32.605298 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:49:32.606363 (+ 1065us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:49:32.606366 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:49:32.607255 (+ 889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:49:32.607255 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:49:32.607515 (+ 260us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:49:32.607519 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 03:49:32.607577 (+ 58us) client_negotiation.cc:240] Negotiation successful
0806 03:49:32.607621 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":655,"thread_start_us":68,"threads_started":1}
I20260806 03:49:32.607844 32675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:49:32.602997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39998 (local address 127.11.253.195:39103)
0806 03:49:32.603456 (+ 459us) server_negotiation.cc:206] Beginning negotiation
0806 03:49:32.603466 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 03:49:32.603474 (+ 8us) server_negotiation.cc:407] Connection header received
0806 03:49:32.603510 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:49:32.603512 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:49:32.603545 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:49:32.603621 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 03:49:32.603928 (+ 307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:49:32.605057 (+ 1129us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:49:32.606326 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:49:32.607054 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:49:32.607132 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:49:32.607189 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0806 03:49:32.607637 (+ 448us) server_negotiation.cc:299] Negotiation successful
0806 03:49:32.607681 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":427,"thread_start_us":90,"threads_started":1}
I20260806 03:49:32.608193 32165 tablet_service.cc:1467] Processing CreateTablet for tablet 1dfe4e16c27e467a88e3aa7a83e27f98 (DEFAULT_TABLE table=test-table [id=e6aa44c02cab46b98bb5361fafa62512]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 03:49:32.608399 32555 tablet_service.cc:1467] Processing CreateTablet for tablet 1dfe4e16c27e467a88e3aa7a83e27f98 (DEFAULT_TABLE table=test-table [id=e6aa44c02cab46b98bb5361fafa62512]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 03:49:32.608515 32165 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1dfe4e16c27e467a88e3aa7a83e27f98. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:49:32.608647 32555 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1dfe4e16c27e467a88e3aa7a83e27f98. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:49:32.610015 32680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:49:32.603242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51988 (local address 127.11.253.194:37941)
0806 03:49:32.603448 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0806 03:49:32.603453 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 03:49:32.603847 (+ 394us) server_negotiation.cc:407] Connection header received
0806 03:49:32.603965 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:49:32.603968 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:49:32.603985 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:49:32.604065 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 03:49:32.604180 (+ 115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:49:32.605194 (+ 1014us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:49:32.606448 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:49:32.607120 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:49:32.607181 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:49:32.607228 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0806 03:49:32.607796 (+ 568us) server_negotiation.cc:299] Negotiation successful
0806 03:49:32.607843 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":165,"thread_start_us":80,"threads_started":1}
I20260806 03:49:32.610723 32356 tablet_service.cc:1467] Processing CreateTablet for tablet 1dfe4e16c27e467a88e3aa7a83e27f98 (DEFAULT_TABLE table=test-table [id=e6aa44c02cab46b98bb5361fafa62512]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 03:49:32.611063 32356 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1dfe4e16c27e467a88e3aa7a83e27f98. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:49:32.611510 32681 tablet_bootstrap.cc:492] T 1dfe4e16c27e467a88e3aa7a83e27f98 P 403aacadd2b546c1b7289fad1f6066eb: Bootstrap starting.
I20260806 03:49:32.612226 32681 tablet_bootstrap.cc:654] T 1dfe4e16c27e467a88e3aa7a83e27f98 P 403aacadd2b546c1b7289fad1f6066eb: Neither blocks nor log segments found. Creating new log.
I20260806 03:49:32.612701 32681 log.cc:824] T 1dfe4e16c27e467a88e3aa7a83e27f98 P 403aacadd2b546c1b7289fad1f6066eb: Log is configured to *not* fsync() on all Append() calls
I20260806 03:49:32.613023 32682 tablet_bootstrap.cc:492] T 1dfe4e16c27e467a88e3aa7a83e27f98 P 67abaec1b40248afb1a33088431ae1d5: Bootstrap starting.
I20260806 03:49:32.613554 32681 tablet_bootstrap.cc:492] T 1dfe4e16c27e467a88e3aa7a83e27f98 P 403aacadd2b546c1b7289fad1f6066eb: No bootstrap required, opened a new log
I20260806 03:49:32.613634 32681 ts_tablet_manager.cc:1397] T 1dfe4e16c27e467a88e3aa7a83e27f98 P 403aacadd2b546c1b7289fad1f6066eb: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 03:49:32.613699 32682 tablet_bootstrap.cc:654] T 1dfe4e16c27e467a88e3aa7a83e27f98 P 67abaec1b40248afb1a33088431ae1d5: Neither blocks nor log segments found. Creating new log.
I20260806 03:49:32.613960 32685 tablet_bootstrap.cc:492] T 1dfe4e16c27e467a88e3aa7a83e27f98 P df657246a4be41cdb398d7c7e988d8e4: Bootstrap starting.
I20260806 03:49:32.614362 32682 log.cc:824] T 1dfe4e16c27e467a88e3aa7a83e27f98 P 67abaec1b40248afb1a33088431ae1d5: Log is configured to *not* fsync() on all Append() calls
I20260806 03:49:32.614697 32685 tablet_bootstrap.cc:654] T 1dfe4e16c27e467a88e3aa7a83e27f98 P df657246a4be41cdb398d7c7e988d8e4: Neither blocks nor log segments found. Creating new log.
I20260806 03:49:32.615196 32682 tablet_bootstrap.cc:492] T 1dfe4e16c27e467a88e3aa7a83e27f98 P 67abaec1b40248afb1a33088431ae1d5: No bootstrap required, opened a new log
I20260806 03:49:32.615200 32685 log.cc:824] T 1dfe4e16c27e467a88e3aa7a83e27f98 P df657246a4be41cdb398d7c7e988d8e4: Log is configured to *not* fsync() on all Append() calls
I20260806 03:49:32.615269 32682 ts_tablet_manager.cc:1397] T 1dfe4e16c27e467a88e3aa7a83e27f98 P 67abaec1b40248afb1a33088431ae1d5: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260806 03:49:32.615888 32685 tablet_bootstrap.cc:492] T 1dfe4e16c27e467a88e3aa7a83e27f98 P df657246a4be41cdb398d7c7e988d8e4: No bootstrap required, opened a new log
I20260806 03:49:32.615764 32681 raft_consensus.cc:348] T 1dfe4e16c27e467a88e3aa7a83e27f98 P 403aacadd2b546c1b7289fad1f6066eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "403aacadd2b546c1b7289fad1f6066eb" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 39103 } } peers { permanent_uuid: "df657246a4be41cdb398d7c7e988d8e4" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 37941 } } peers { permanent_uuid: "67abaec1b40248afb1a33088431ae1d5" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 36851 } }
I20260806 03:49:32.615952 32681 raft_consensus.cc:374] T 1dfe4e16c27e467a88e3aa7a83e27f98 P 403aacadd2b546c1b7289fad1f6066eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:49:32.615980 32685 ts_tablet_manager.cc:1397] T 1dfe4e16c27e467a88e3aa7a83e27f98 P df657246a4be41cdb398d7c7e988d8e4: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 03:49:32.616001 32681 raft_consensus.cc:729] T 1dfe4e16c27e467a88e3aa7a83e27f98 P 403aacadd2b546c1b7289fad1f6066eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 403aacadd2b546c1b7289fad1f6066eb, State: Initialized, Role: FOLLOWER
I20260806 03:49:32.616146 32681 consensus_queue.cc:260] T 1dfe4e16c27e467a88e3aa7a83e27f98 P 403aacadd2b546c1b7289fad1f6066eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "403aacadd2b546c1b7289fad1f6066eb" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 39103 } } peers { permanent_uuid: "df657246a4be41cdb398d7c7e988d8e4" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 37941 } } peers { permanent_uuid: "67abaec1b40248afb1a33088431ae1d5" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 36851 } }
I20260806 03:49:32.616443 32681 ts_tablet_manager.cc:1428] T 1dfe4e16c27e467a88e3aa7a83e27f98 P 403aacadd2b546c1b7289fad1f6066eb: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 03:49:32.616570 32656 heartbeater.cc:503] Master 127.11.253.254:44939 was elected leader, sending a full tablet report...
I20260806 03:49:32.617609 32682 raft_consensus.cc:348] T 1dfe4e16c27e467a88e3aa7a83e27f98 P 67abaec1b40248afb1a33088431ae1d5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "403aacadd2b546c1b7289fad1f6066eb" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 39103 } } peers { permanent_uuid: "df657246a4be41cdb398d7c7e988d8e4" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 37941 } } peers { permanent_uuid: "67abaec1b40248afb1a33088431ae1d5" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 36851 } }
I20260806 03:49:32.617746 32682 raft_consensus.cc:374] T 1dfe4e16c27e467a88e3aa7a83e27f98 P 67abaec1b40248afb1a33088431ae1d5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:49:32.617769 32682 raft_consensus.cc:729] T 1dfe4e16c27e467a88e3aa7a83e27f98 P 67abaec1b40248afb1a33088431ae1d5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 67abaec1b40248afb1a33088431ae1d5, State: Initialized, Role: FOLLOWER
I20260806 03:49:32.617933 32682 consensus_queue.cc:260] T 1dfe4e16c27e467a88e3aa7a83e27f98 P 67abaec1b40248afb1a33088431ae1d5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "403aacadd2b546c1b7289fad1f6066eb" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 39103 } } peers { permanent_uuid: "df657246a4be41cdb398d7c7e988d8e4" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 37941 } } peers { permanent_uuid: "67abaec1b40248afb1a33088431ae1d5" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 36851 } }
I20260806 03:49:32.617928 32685 raft_consensus.cc:348] T 1dfe4e16c27e467a88e3aa7a83e27f98 P df657246a4be41cdb398d7c7e988d8e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "403aacadd2b546c1b7289fad1f6066eb" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 39103 } } peers { permanent_uuid: "df657246a4be41cdb398d7c7e988d8e4" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 37941 } } peers { permanent_uuid: "67abaec1b40248afb1a33088431ae1d5" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 36851 } }
I20260806 03:49:32.618112 32685 raft_consensus.cc:374] T 1dfe4e16c27e467a88e3aa7a83e27f98 P df657246a4be41cdb398d7c7e988d8e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:49:32.618167 32685 raft_consensus.cc:729] T 1dfe4e16c27e467a88e3aa7a83e27f98 P df657246a4be41cdb398d7c7e988d8e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: df657246a4be41cdb398d7c7e988d8e4, State: Initialized, Role: FOLLOWER
I20260806 03:49:32.618247 32682 ts_tablet_manager.cc:1428] T 1dfe4e16c27e467a88e3aa7a83e27f98 P 67abaec1b40248afb1a33088431ae1d5: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260806 03:49:32.618384 32258 heartbeater.cc:503] Master 127.11.253.254:44939 was elected leader, sending a full tablet report...
I20260806 03:49:32.618305 32685 consensus_queue.cc:260] T 1dfe4e16c27e467a88e3aa7a83e27f98 P df657246a4be41cdb398d7c7e988d8e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "403aacadd2b546c1b7289fad1f6066eb" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 39103 } } peers { permanent_uuid: "df657246a4be41cdb398d7c7e988d8e4" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 37941 } } peers { permanent_uuid: "67abaec1b40248afb1a33088431ae1d5" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 36851 } }
I20260806 03:49:32.618947 32685 ts_tablet_manager.cc:1428] T 1dfe4e16c27e467a88e3aa7a83e27f98 P df657246a4be41cdb398d7c7e988d8e4: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 03:49:32.619014 32450 heartbeater.cc:503] Master 127.11.253.254:44939 was elected leader, sending a full tablet report...
W20260806 03:49:32.665561 32455 tablet.cc:2369] T 1dfe4e16c27e467a88e3aa7a83e27f98 P df657246a4be41cdb398d7c7e988d8e4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 03:49:32.685755 32691 raft_consensus.cc:482] T 1dfe4e16c27e467a88e3aa7a83e27f98 P 403aacadd2b546c1b7289fad1f6066eb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 03:49:32.685923 32691 raft_consensus.cc:504] T 1dfe4e16c27e467a88e3aa7a83e27f98 P 403aacadd2b546c1b7289fad1f6066eb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "403aacadd2b546c1b7289fad1f6066eb" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 39103 } } peers { permanent_uuid: "df657246a4be41cdb398d7c7e988d8e4" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 37941 } } peers { permanent_uuid: "67abaec1b40248afb1a33088431ae1d5" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 36851 } }
I20260806 03:49:32.686409 32691 leader_election.cc:288] T 1dfe4e16c27e467a88e3aa7a83e27f98 P 403aacadd2b546c1b7289fad1f6066eb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers df657246a4be41cdb398d7c7e988d8e4 (127.11.253.194:37941), 67abaec1b40248afb1a33088431ae1d5 (127.11.253.193:36851)
I20260806 03:49:32.689424 32658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:49:32.686601 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.193:36851 (local address 127.11.253.195:57827)
0806 03:49:32.686667 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0806 03:49:32.686680 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 03:49:32.686742 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:49:32.686980 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:49:32.686981 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:49:32.686984 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 03:49:32.687092 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:49:32.687094 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:49:32.687854 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:49:32.687855 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:49:32.688603 (+ 748us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:49:32.688605 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:49:32.689188 (+ 583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:49:32.689189 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:49:32.689273 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:49:32.689277 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 03:49:32.689335 (+ 58us) client_negotiation.cc:240] Negotiation successful
0806 03:49:32.689366 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46,"mutex_wait_us":2}
I20260806 03:49:32.689574 32679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:49:32.686703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.253.195:57827 (local address 127.11.253.193:36851)
0806 03:49:32.686773 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0806 03:49:32.686776 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 03:49:32.686790 (+ 14us) server_negotiation.cc:407] Connection header received
0806 03:49:32.686854 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:49:32.686855 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:49:32.686866 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:49:32.686941 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 03:49:32.687181 (+ 240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:49:32.687729 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:49:32.688700 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:49:32.689102 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:49:32.689160 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:49:32.689168 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0806 03:49:32.689451 (+ 283us) server_negotiation.cc:299] Negotiation successful
0806 03:49:32.689502 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260806 03:49:32.689599 32694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:49:32.686601 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.194:37941 (local address 127.11.253.195:42879)
0806 03:49:32.686858 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0806 03:49:32.686866 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0806 03:49:32.686937 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:49:32.687134 (+ 197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:49:32.687135 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:49:32.687136 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 03:49:32.687203 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:49:32.687204 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:49:32.687895 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:49:32.687896 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:49:32.688626 (+ 730us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:49:32.688628 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:49:32.689249 (+ 621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:49:32.689249 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:49:32.689303 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:49:32.689306 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 03:49:32.689485 (+ 179us) client_negotiation.cc:240] Negotiation successful
0806 03:49:32.689522 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":244,"mutex_wait_us":22,"thread_start_us":132,"threads_started":1}
I20260806 03:49:32.689713 32680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:49:32.686702 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.253.195:42879 (local address 127.11.253.194:37941)
0806 03:49:32.686847 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0806 03:49:32.686849 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:49:32.686928 (+ 79us) server_negotiation.cc:407] Connection header received
0806 03:49:32.687023 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:49:32.687024 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:49:32.687037 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:49:32.687068 (+ 31us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 03:49:32.687274 (+ 206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:49:32.687757 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:49:32.688755 (+ 998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:49:32.689124 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:49:32.689219 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:49:32.689227 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0806 03:49:32.689601 (+ 374us) server_negotiation.cc:299] Negotiation successful
0806 03:49:32.689644 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122}
I20260806 03:49:32.690063 32382 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1dfe4e16c27e467a88e3aa7a83e27f98" candidate_uuid: "403aacadd2b546c1b7289fad1f6066eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "df657246a4be41cdb398d7c7e988d8e4" is_pre_election: true
I20260806 03:49:32.690268 32382 raft_consensus.cc:2454] T 1dfe4e16c27e467a88e3aa7a83e27f98 P df657246a4be41cdb398d7c7e988d8e4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 403aacadd2b546c1b7289fad1f6066eb in term 0.
I20260806 03:49:32.690385 32194 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1dfe4e16c27e467a88e3aa7a83e27f98" candidate_uuid: "403aacadd2b546c1b7289fad1f6066eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "67abaec1b40248afb1a33088431ae1d5" is_pre_election: true
I20260806 03:49:32.690514 32484 leader_election.cc:302] T 1dfe4e16c27e467a88e3aa7a83e27f98 P 403aacadd2b546c1b7289fad1f6066eb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 403aacadd2b546c1b7289fad1f6066eb, df657246a4be41cdb398d7c7e988d8e4; no voters:
I20260806 03:49:32.690554 32194 raft_consensus.cc:2454] T 1dfe4e16c27e467a88e3aa7a83e27f98 P 67abaec1b40248afb1a33088431ae1d5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 403aacadd2b546c1b7289fad1f6066eb in term 0.
I20260806 03:49:32.690644 32691 raft_consensus.cc:2781] T 1dfe4e16c27e467a88e3aa7a83e27f98 P 403aacadd2b546c1b7289fad1f6066eb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 03:49:32.690748 32691 raft_consensus.cc:482] T 1dfe4e16c27e467a88e3aa7a83e27f98 P 403aacadd2b546c1b7289fad1f6066eb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 03:49:32.690797 32691 raft_consensus.cc:3037] T 1dfe4e16c27e467a88e3aa7a83e27f98 P 403aacadd2b546c1b7289fad1f6066eb [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:49:32.691606 32691 raft_consensus.cc:504] T 1dfe4e16c27e467a88e3aa7a83e27f98 P 403aacadd2b546c1b7289fad1f6066eb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "403aacadd2b546c1b7289fad1f6066eb" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 39103 } } peers { permanent_uuid: "df657246a4be41cdb398d7c7e988d8e4" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 37941 } } peers { permanent_uuid: "67abaec1b40248afb1a33088431ae1d5" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 36851 } }
I20260806 03:49:32.691793 32691 leader_election.cc:288] T 1dfe4e16c27e467a88e3aa7a83e27f98 P 403aacadd2b546c1b7289fad1f6066eb [CANDIDATE]: Term 1 election: Requested vote from peers df657246a4be41cdb398d7c7e988d8e4 (127.11.253.194:37941), 67abaec1b40248afb1a33088431ae1d5 (127.11.253.193:36851)
I20260806 03:49:32.691932 32382 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1dfe4e16c27e467a88e3aa7a83e27f98" candidate_uuid: "403aacadd2b546c1b7289fad1f6066eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "df657246a4be41cdb398d7c7e988d8e4"
I20260806 03:49:32.691977 32194 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1dfe4e16c27e467a88e3aa7a83e27f98" candidate_uuid: "403aacadd2b546c1b7289fad1f6066eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "67abaec1b40248afb1a33088431ae1d5"
I20260806 03:49:32.692018 32382 raft_consensus.cc:3037] T 1dfe4e16c27e467a88e3aa7a83e27f98 P df657246a4be41cdb398d7c7e988d8e4 [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:49:32.692060 32194 raft_consensus.cc:3037] T 1dfe4e16c27e467a88e3aa7a83e27f98 P 67abaec1b40248afb1a33088431ae1d5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:49:32.692850 32382 raft_consensus.cc:2454] T 1dfe4e16c27e467a88e3aa7a83e27f98 P df657246a4be41cdb398d7c7e988d8e4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 403aacadd2b546c1b7289fad1f6066eb in term 1.
I20260806 03:49:32.692878 32194 raft_consensus.cc:2454] T 1dfe4e16c27e467a88e3aa7a83e27f98 P 67abaec1b40248afb1a33088431ae1d5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 403aacadd2b546c1b7289fad1f6066eb in term 1.
I20260806 03:49:32.693079 32484 leader_election.cc:302] T 1dfe4e16c27e467a88e3aa7a83e27f98 P 403aacadd2b546c1b7289fad1f6066eb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 403aacadd2b546c1b7289fad1f6066eb, df657246a4be41cdb398d7c7e988d8e4; no voters:
I20260806 03:49:32.693214 32691 raft_consensus.cc:2781] T 1dfe4e16c27e467a88e3aa7a83e27f98 P 403aacadd2b546c1b7289fad1f6066eb [term 1 FOLLOWER]: Leader election won for term 1
I20260806 03:49:32.693384 32691 raft_consensus.cc:686] T 1dfe4e16c27e467a88e3aa7a83e27f98 P 403aacadd2b546c1b7289fad1f6066eb [term 1 LEADER]: Becoming Leader. State: Replica: 403aacadd2b546c1b7289fad1f6066eb, State: Running, Role: LEADER
I20260806 03:49:32.693624 32691 consensus_queue.cc:237] T 1dfe4e16c27e467a88e3aa7a83e27f98 P 403aacadd2b546c1b7289fad1f6066eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "403aacadd2b546c1b7289fad1f6066eb" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 39103 } } peers { permanent_uuid: "df657246a4be41cdb398d7c7e988d8e4" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 37941 } } peers { permanent_uuid: "67abaec1b40248afb1a33088431ae1d5" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 36851 } }
I20260806 03:49:32.694589 31982 catalog_manager.cc:5495] T 1dfe4e16c27e467a88e3aa7a83e27f98 P 403aacadd2b546c1b7289fad1f6066eb reported cstate change: term changed from 0 to 1, leader changed from <none> to 403aacadd2b546c1b7289fad1f6066eb (127.11.253.195). New cstate: current_term: 1 leader_uuid: "403aacadd2b546c1b7289fad1f6066eb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "403aacadd2b546c1b7289fad1f6066eb" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 39103 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "df657246a4be41cdb398d7c7e988d8e4" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 37941 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "67abaec1b40248afb1a33088431ae1d5" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 36851 } health_report { overall_health: UNKNOWN } } }
I20260806 03:49:32.722813 32675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:49:32.720410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40002 (local address 127.11.253.195:39103)
0806 03:49:32.720479 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0806 03:49:32.720481 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:49:32.720490 (+ 9us) server_negotiation.cc:407] Connection header received
0806 03:49:32.720537 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:49:32.720538 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:49:32.720556 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:49:32.720620 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 03:49:32.720768 (+ 148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:49:32.721307 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:49:32.722058 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:49:32.722241 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:49:32.722296 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:49:32.722436 (+ 140us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 03:49:32.722539 (+ 103us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 03:49:32.722602 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0806 03:49:32.722657 (+ 55us) server_negotiation.cc:299] Negotiation successful
0806 03:49:32.722676 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260806 03:49:32.724882 32194 raft_consensus.cc:1261] T 1dfe4e16c27e467a88e3aa7a83e27f98 P 67abaec1b40248afb1a33088431ae1d5 [term 1 FOLLOWER]: Refusing update from remote peer 403aacadd2b546c1b7289fad1f6066eb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 03:49:32.724936 32382 raft_consensus.cc:1261] T 1dfe4e16c27e467a88e3aa7a83e27f98 P df657246a4be41cdb398d7c7e988d8e4 [term 1 FOLLOWER]: Refusing update from remote peer 403aacadd2b546c1b7289fad1f6066eb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 03:49:32.725257 32695 consensus_queue.cc:1035] T 1dfe4e16c27e467a88e3aa7a83e27f98 P 403aacadd2b546c1b7289fad1f6066eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "67abaec1b40248afb1a33088431ae1d5" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 36851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 03:49:32.725394 32691 consensus_queue.cc:1035] T 1dfe4e16c27e467a88e3aa7a83e27f98 P 403aacadd2b546c1b7289fad1f6066eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "df657246a4be41cdb398d7c7e988d8e4" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 37941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 03:49:32.727955 32699 mvcc.cc:205] Tried to move back new op lower bound from 7315407555477921792 to 7315407555354124288. Current Snapshot: MvccSnapshot[applied={T|T < 7315407555477921792}]
I20260806 03:49:32.728574 32700 mvcc.cc:205] Tried to move back new op lower bound from 7315407555477921792 to 7315407555354124288. Current Snapshot: MvccSnapshot[applied={T|T < 7315407555477921792}]
I20260806 03:49:32.730046 31983 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36800:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 03:49:32.730124 31983 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36800:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 03:49:32.730942 31983 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9c6e110e6ae045d2992beccbc2b49594: Sending DeleteTablet for 3 replicas of tablet 1dfe4e16c27e467a88e3aa7a83e27f98
I20260806 03:49:32.731249 32555 tablet_service.cc:1514] Processing DeleteTablet for tablet 1dfe4e16c27e467a88e3aa7a83e27f98 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 03:49:32 UTC) from {username='kudu', principal='kudu/127.11.253.254@KRBTEST.COM'} at 127.0.0.1:39998
I20260806 03:49:32.731295 32356 tablet_service.cc:1514] Processing DeleteTablet for tablet 1dfe4e16c27e467a88e3aa7a83e27f98 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 03:49:32 UTC) from {username='kudu', principal='kudu/127.11.253.254@KRBTEST.COM'} at 127.0.0.1:51988
I20260806 03:49:32.731330 32165 tablet_service.cc:1514] Processing DeleteTablet for tablet 1dfe4e16c27e467a88e3aa7a83e27f98 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 03:49:32 UTC) from {username='kudu', principal='kudu/127.11.253.254@KRBTEST.COM'} at 127.0.0.1:48746
I20260806 03:49:32.731483 32714 tablet_replica.cc:330] T 1dfe4e16c27e467a88e3aa7a83e27f98 P df657246a4be41cdb398d7c7e988d8e4: stopping tablet replica
I20260806 03:49:32.731563 32715 tablet_replica.cc:330] T 1dfe4e16c27e467a88e3aa7a83e27f98 P 67abaec1b40248afb1a33088431ae1d5: stopping tablet replica
I20260806 03:49:32.731583 32714 raft_consensus.cc:2229] T 1dfe4e16c27e467a88e3aa7a83e27f98 P df657246a4be41cdb398d7c7e988d8e4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 03:49:32.731643 32715 raft_consensus.cc:2229] T 1dfe4e16c27e467a88e3aa7a83e27f98 P 67abaec1b40248afb1a33088431ae1d5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 03:49:32.731680 32714 raft_consensus.cc:2258] T 1dfe4e16c27e467a88e3aa7a83e27f98 P df657246a4be41cdb398d7c7e988d8e4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 03:49:32.731671 12279 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskToVWRe/build/release/bin/kudu with pid 32076
I20260806 03:49:32.731721 32715 raft_consensus.cc:2258] T 1dfe4e16c27e467a88e3aa7a83e27f98 P 67abaec1b40248afb1a33088431ae1d5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 03:49:32.739399 32713 tablet_replica.cc:330] T 1dfe4e16c27e467a88e3aa7a83e27f98 P 403aacadd2b546c1b7289fad1f6066eb: stopping tablet replica
I20260806 03:49:32.739535 32713 raft_consensus.cc:2229] T 1dfe4e16c27e467a88e3aa7a83e27f98 P 403aacadd2b546c1b7289fad1f6066eb [term 1 LEADER]: Raft consensus shutting down.
I20260806 03:49:32.739637 32713 raft_consensus.cc:2258] T 1dfe4e16c27e467a88e3aa7a83e27f98 P 403aacadd2b546c1b7289fad1f6066eb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 03:49:32.741026 32714 ts_tablet_manager.cc:1905] T 1dfe4e16c27e467a88e3aa7a83e27f98 P df657246a4be41cdb398d7c7e988d8e4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 03:49:32.742871 32713 ts_tablet_manager.cc:1905] T 1dfe4e16c27e467a88e3aa7a83e27f98 P 403aacadd2b546c1b7289fad1f6066eb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 03:49:32.745362 32713 ts_tablet_manager.cc:1918] T 1dfe4e16c27e467a88e3aa7a83e27f98 P 403aacadd2b546c1b7289fad1f6066eb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 03:49:32.745471 32713 log.cc:1196] T 1dfe4e16c27e467a88e3aa7a83e27f98 P 403aacadd2b546c1b7289fad1f6066eb: Deleting WAL directory at /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/minicluster-data/ts-2/wal/wals/1dfe4e16c27e467a88e3aa7a83e27f98
I20260806 03:49:32.745863 32713 ts_tablet_manager.cc:1939] T 1dfe4e16c27e467a88e3aa7a83e27f98 P 403aacadd2b546c1b7289fad1f6066eb: Deleting consensus metadata
I20260806 03:49:32.746498 31960 catalog_manager.cc:4841] TS 403aacadd2b546c1b7289fad1f6066eb (127.11.253.195:39103): tablet 1dfe4e16c27e467a88e3aa7a83e27f98 (table test-table [id=e6aa44c02cab46b98bb5361fafa62512]) successfully deleted
I20260806 03:49:32.748340 32714 ts_tablet_manager.cc:1918] T 1dfe4e16c27e467a88e3aa7a83e27f98 P df657246a4be41cdb398d7c7e988d8e4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 03:49:32.748458 32714 log.cc:1196] T 1dfe4e16c27e467a88e3aa7a83e27f98 P df657246a4be41cdb398d7c7e988d8e4: Deleting WAL directory at /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/minicluster-data/ts-1/wal/wals/1dfe4e16c27e467a88e3aa7a83e27f98
I20260806 03:49:32.748834 32714 ts_tablet_manager.cc:1939] T 1dfe4e16c27e467a88e3aa7a83e27f98 P df657246a4be41cdb398d7c7e988d8e4: Deleting consensus metadata
I20260806 03:49:32.749436 31960 catalog_manager.cc:4841] TS df657246a4be41cdb398d7c7e988d8e4 (127.11.253.194:37941): tablet 1dfe4e16c27e467a88e3aa7a83e27f98 (table test-table [id=e6aa44c02cab46b98bb5361fafa62512]) successfully deleted
W20260806 03:49:32.757956 31961 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.253.193:36851 (error 108)
I20260806 03:49:32.758208 12279 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskToVWRe/build/release/bin/kudu with pid 32266
I20260806 03:49:32.764566 32670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:49:32.764421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.193:36851 (local address 127.0.0.1:48754)
0806 03:49:32.764483 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0806 03:49:32.764517 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.253.193:36851: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260806 03:49:32.764689 31961 catalog_manager.cc:4584] TS 67abaec1b40248afb1a33088431ae1d5 (127.11.253.193:36851): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 1dfe4e16c27e467a88e3aa7a83e27f98: Network error: Client connection negotiation failed: client connection to 127.11.253.193:36851: connect: Connection refused (error 111)
I20260806 03:49:32.776978 12279 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskToVWRe/build/release/bin/kudu with pid 32459
I20260806 03:49:32.787341 12279 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskToVWRe/build/release/bin/kudu with pid 31937
I20260806 03:49:32.796738 12279 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskToVWRe/build/release/bin/kudu
/tmp/dist-test-taskToVWRe/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.253.254:44939
--webserver_interface=127.11.253.254
--webserver_port=38907
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.254.keytab
--principal=kudu/127.11.253.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.253.212:40573
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.253.254:44939
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:49:32.884763 32719 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:49:32.885021 32719 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:49:32.885047 32719 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:49:32.886343 32719 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 03:49:32.886392 32719 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:49:32.886409 32719 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:49:32.886425 32719 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 03:49:32.886437 32719 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 03:49:32.888134 32719 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.253.212:40573
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.11.253.254:44939
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.254.keytab
--principal=kudu/127.11.253.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.253.254:44939
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.253.254
--webserver_port=38907
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32719
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:49:32.888414 32719 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:49:32.888698 32719 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:49:32.899402 32731 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:49:32.899402 32729 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:49:32.899372 32728 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:49:32.899442 32719 server_base.cc:1034] running on GCE node
I20260806 03:49:32.899892 32719 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:49:32.900243 32719 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:49:32.901427 32719 hybrid_clock.cc:648] HybridClock initialized: now 1785988172901416 us; error 52 us; skew 500 ppm
Aug 06 03:49:32 dist-test-slave-6ntk krb5kdc[31917](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988172, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.253.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:49:32.904261 32719 init.cc:377] Logged in from keytab as kudu/127.11.253.254@KRBTEST.COM (short username kudu)
I20260806 03:49:32.904764 32719 webserver.cc:460] Webserver started at http://127.11.253.254:38907/ using document root <none> and password file <none>
I20260806 03:49:32.904999 32719 fs_manager.cc:359] Metadata directory not provided
I20260806 03:49:32.905057 32719 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:49:32.906579 32719 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 03:49:32.907374 32739 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:49:32.907635 32719 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 03:49:32.907747 32719 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/minicluster-data/master-0/data,/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/minicluster-data/master-0/wal
uuid: "9c6e110e6ae045d2992beccbc2b49594"
format_stamp: "Formatted at 2026-08-06 03:49:31 on dist-test-slave-6ntk"
server_key: "c1b753a64abc5121b494e9ad05f3ce2c"
server_key_iv: "dbed1853eafb71a198456bd471eec18d"
server_key_version: "encryptionkey@0"
I20260806 03:49:32.908170 32719 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:49:32.945232 32719 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:49:32.947794 32719 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:49:32.947965 32719 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:49:32.952560 32719 rpc_server.cc:307] RPC server started. Bound to: 127.11.253.254:44939
I20260806 03:49:32.952663 352 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.253.254:44939 every 8 connection(s)
I20260806 03:49:32.953060 32719 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/minicluster-data/master-0/data/info.pb
I20260806 03:49:32.954963 353 sys_catalog.cc:263] Verifying existing consensus state
I20260806 03:49:32.955762 353 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9c6e110e6ae045d2992beccbc2b49594: Bootstrap starting.
I20260806 03:49:32.963212 12279 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskToVWRe/build/release/bin/kudu as pid 32719
I20260806 03:49:32.963541 12279 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskToVWRe/build/release/bin/kudu
/tmp/dist-test-taskToVWRe/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.253.193:36851
--local_ip_for_outbound_sockets=127.11.253.193
--tserver_master_addrs=127.11.253.254:44939
--webserver_port=36067
--webserver_interface=127.11.253.193
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.193.keytab
--principal=kudu/127.11.253.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.253.212:40573
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 03:49:32.965965 353 log.cc:824] T 00000000000000000000000000000000 P 9c6e110e6ae045d2992beccbc2b49594: Log is configured to *not* fsync() on all Append() calls
I20260806 03:49:32.968123 353 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9c6e110e6ae045d2992beccbc2b49594: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 03:49:32.968482 353 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9c6e110e6ae045d2992beccbc2b49594: Bootstrap complete.
I20260806 03:49:32.971194 353 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9c6e110e6ae045d2992beccbc2b49594 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c6e110e6ae045d2992beccbc2b49594" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 44939 } }
I20260806 03:49:32.971470 353 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9c6e110e6ae045d2992beccbc2b49594 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c6e110e6ae045d2992beccbc2b49594, State: Initialized, Role: FOLLOWER
I20260806 03:49:32.971680 353 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9c6e110e6ae045d2992beccbc2b49594 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c6e110e6ae045d2992beccbc2b49594" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 44939 } }
I20260806 03:49:32.971778 353 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9c6e110e6ae045d2992beccbc2b49594 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 03:49:32.971850 353 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9c6e110e6ae045d2992beccbc2b49594 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 03:49:32.971912 353 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9c6e110e6ae045d2992beccbc2b49594 [term 1 FOLLOWER]: Advancing to term 2
I20260806 03:49:32.973047 353 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9c6e110e6ae045d2992beccbc2b49594 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c6e110e6ae045d2992beccbc2b49594" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 44939 } }
I20260806 03:49:32.973186 353 leader_election.cc:302] T 00000000000000000000000000000000 P 9c6e110e6ae045d2992beccbc2b49594 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9c6e110e6ae045d2992beccbc2b49594; no voters:
I20260806 03:49:32.973415 353 leader_election.cc:288] T 00000000000000000000000000000000 P 9c6e110e6ae045d2992beccbc2b49594 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260806 03:49:32.973459 358 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9c6e110e6ae045d2992beccbc2b49594 [term 2 FOLLOWER]: Leader election won for term 2
I20260806 03:49:32.973587 358 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9c6e110e6ae045d2992beccbc2b49594 [term 2 LEADER]: Becoming Leader. State: Replica: 9c6e110e6ae045d2992beccbc2b49594, State: Running, Role: LEADER
I20260806 03:49:32.973687 358 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9c6e110e6ae045d2992beccbc2b49594 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c6e110e6ae045d2992beccbc2b49594" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 44939 } }
I20260806 03:49:32.973742 353 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9c6e110e6ae045d2992beccbc2b49594 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 03:49:32.974053 358 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9c6e110e6ae045d2992beccbc2b49594 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9c6e110e6ae045d2992beccbc2b49594. Latest consensus state: current_term: 2 leader_uuid: "9c6e110e6ae045d2992beccbc2b49594" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c6e110e6ae045d2992beccbc2b49594" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 44939 } } }
I20260806 03:49:32.974133 358 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9c6e110e6ae045d2992beccbc2b49594 [sys.catalog]: This master's current role is: LEADER
I20260806 03:49:32.974318 361 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9c6e110e6ae045d2992beccbc2b49594 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "9c6e110e6ae045d2992beccbc2b49594" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c6e110e6ae045d2992beccbc2b49594" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 44939 } } }
I20260806 03:49:32.974401 361 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9c6e110e6ae045d2992beccbc2b49594 [sys.catalog]: This master's current role is: LEADER
I20260806 03:49:32.974864 368 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 03:49:32.975801 368 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 03:49:32.975932 368 catalog_manager.cc:1259] Loaded cluster ID: b5c142d5173b48c8b1eaa358a27de72b
I20260806 03:49:32.976243 368 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 03:49:32.976526 368 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260806 03:49:32.979647 368 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260806 03:49:32.981709 368 catalog_manager.cc:1504] Loading token signing keys...
I20260806 03:49:32.981980 368 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 9c6e110e6ae045d2992beccbc2b49594: Loaded TSK: 0
I20260806 03:49:32.982246 368 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 03:49:33.066301 355 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:49:33.066556 355 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:49:33.066601 355 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:49:33.067823 355 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:49:33.067904 355 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:49:33.067960 355 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.253.193
I20260806 03:49:33.069620 355 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.253.212:40573
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.193.keytab
--principal=kudu/127.11.253.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.253.193:36851
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.253.193
--webserver_port=36067
--tserver_master_addrs=127.11.253.254:44939
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.355
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.253.193
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:49:33.069940 355 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:49:33.070223 355 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:49:33.076613 398 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:49:33.076623 397 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:49:33.076663 400 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:49:33.077301 355 server_base.cc:1034] running on GCE node
I20260806 03:49:33.077564 355 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:49:33.077934 355 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:49:33.079144 355 hybrid_clock.cc:648] HybridClock initialized: now 1785988173079123 us; error 39 us; skew 500 ppm
Aug 06 03:49:33 dist-test-slave-6ntk krb5kdc[31917](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988173, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.253.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:49:33.082067 355 init.cc:377] Logged in from keytab as kudu/127.11.253.193@KRBTEST.COM (short username kudu)
I20260806 03:49:33.082619 355 webserver.cc:460] Webserver started at http://127.11.253.193:36067/ using document root <none> and password file <none>
I20260806 03:49:33.082906 355 fs_manager.cc:359] Metadata directory not provided
I20260806 03:49:33.082968 355 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:49:33.084444 355 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 03:49:33.085191 408 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:49:33.085392 355 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 03:49:33.085489 355 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/minicluster-data/ts-0/data,/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/minicluster-data/ts-0/wal
uuid: "67abaec1b40248afb1a33088431ae1d5"
format_stamp: "Formatted at 2026-08-06 03:49:32 on dist-test-slave-6ntk"
server_key: "97e4da2295ff60317613732a64c0f93b"
server_key_iv: "94348828464dd14f650c05dac0d9e775"
server_key_version: "encryptionkey@0"
I20260806 03:49:33.085917 355 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:49:33.103487 355 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:49:33.106144 355 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:49:33.106325 355 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:49:33.106616 355 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 03:49:33.107270 416 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260806 03:49:33.108852 355 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260806 03:49:33.108906 355 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260806 03:49:33.108943 355 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260806 03:49:33.109654 355 ts_tablet_manager.cc:610] Registered 1 tablets
I20260806 03:49:33.109702 355 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260806 03:49:33.109763 416 tablet_bootstrap.cc:492] T 1dfe4e16c27e467a88e3aa7a83e27f98 P 67abaec1b40248afb1a33088431ae1d5: Bootstrap starting.
I20260806 03:49:33.117687 355 rpc_server.cc:307] RPC server started. Bound to: 127.11.253.193:36851
I20260806 03:49:33.117916 570 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.253.193:36851 every 8 connection(s)
I20260806 03:49:33.118274 355 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/minicluster-data/ts-0/data/info.pb
I20260806 03:49:33.120267 12279 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskToVWRe/build/release/bin/kudu as pid 355
I20260806 03:49:33.120628 12279 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskToVWRe/build/release/bin/kudu
/tmp/dist-test-taskToVWRe/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.253.194:37941
--local_ip_for_outbound_sockets=127.11.253.194
--tserver_master_addrs=127.11.253.254:44939
--webserver_port=39759
--webserver_interface=127.11.253.194
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.194.keytab
--principal=kudu/127.11.253.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.253.212:40573
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 03:49:33.122592 416 log.cc:824] T 1dfe4e16c27e467a88e3aa7a83e27f98 P 67abaec1b40248afb1a33088431ae1d5: Log is configured to *not* fsync() on all Append() calls
I20260806 03:49:33.125231 416 tablet_bootstrap.cc:492] T 1dfe4e16c27e467a88e3aa7a83e27f98 P 67abaec1b40248afb1a33088431ae1d5: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 03:49:33.125545 416 tablet_bootstrap.cc:492] T 1dfe4e16c27e467a88e3aa7a83e27f98 P 67abaec1b40248afb1a33088431ae1d5: Bootstrap complete.
I20260806 03:49:33.126756 416 ts_tablet_manager.cc:1397] T 1dfe4e16c27e467a88e3aa7a83e27f98 P 67abaec1b40248afb1a33088431ae1d5: Time spent bootstrapping tablet: real 0.017s user 0.002s sys 0.013s
Aug 06 03:49:33 dist-test-slave-6ntk krb5kdc[31917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988173, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.253.193@KRBTEST.COM for kudu/127.11.253.254@KRBTEST.COM
I20260806 03:49:33.128544 416 raft_consensus.cc:348] T 1dfe4e16c27e467a88e3aa7a83e27f98 P 67abaec1b40248afb1a33088431ae1d5 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "403aacadd2b546c1b7289fad1f6066eb" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 39103 } } peers { permanent_uuid: "df657246a4be41cdb398d7c7e988d8e4" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 37941 } } peers { permanent_uuid: "67abaec1b40248afb1a33088431ae1d5" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 36851 } }
I20260806 03:49:33.128958 416 raft_consensus.cc:729] T 1dfe4e16c27e467a88e3aa7a83e27f98 P 67abaec1b40248afb1a33088431ae1d5 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 67abaec1b40248afb1a33088431ae1d5, State: Initialized, Role: FOLLOWER
I20260806 03:49:33.129142 416 consensus_queue.cc:260] T 1dfe4e16c27e467a88e3aa7a83e27f98 P 67abaec1b40248afb1a33088431ae1d5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "403aacadd2b546c1b7289fad1f6066eb" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 39103 } } peers { permanent_uuid: "df657246a4be41cdb398d7c7e988d8e4" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 37941 } } peers { permanent_uuid: "67abaec1b40248afb1a33088431ae1d5" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 36851 } }
I20260806 03:49:33.129473 416 ts_tablet_manager.cc:1428] T 1dfe4e16c27e467a88e3aa7a83e27f98 P 67abaec1b40248afb1a33088431ae1d5: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260806 03:49:33.132057 574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:49:33.119038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:44939 (local address 127.11.253.193:46651)
0806 03:49:33.119263 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0806 03:49:33.119314 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0806 03:49:33.119887 (+ 573us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:49:33.122446 (+ 2559us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:49:33.122450 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:49:33.122862 (+ 412us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:49:33.123145 (+ 283us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:49:33.123152 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:49:33.124254 (+ 1102us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:49:33.124255 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:49:33.125131 (+ 876us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:49:33.125133 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:49:33.125841 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:49:33.125842 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:49:33.126011 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:49:33.126631 (+ 620us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:49:33.126640 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:49:33.128204 (+ 1564us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:49:33.130388 (+ 2184us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:49:33.130397 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:49:33.130399 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:49:33.130634 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:49:33.130910 (+ 276us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:49:33.130911 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:49:33.130911 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:49:33.131023 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:49:33.131421 (+ 398us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:49:33.131429 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:49:33.131564 (+ 135us) client_negotiation.cc:769] Sending connection context
0806 03:49:33.131673 (+ 109us) client_negotiation.cc:240] Negotiation successful
0806 03:49:33.131862 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":124,"threads_started":1}
I20260806 03:49:33.132057 575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:49:33.119186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.253.193:46651 (local address 127.11.253.254:44939)
0806 03:49:33.122027 (+ 2841us) server_negotiation.cc:206] Beginning negotiation
0806 03:49:33.122036 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 03:49:33.122073 (+ 37us) server_negotiation.cc:407] Connection header received
0806 03:49:33.122184 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:49:33.122190 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:49:33.122233 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:49:33.122419 (+ 186us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:49:33.123265 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:49:33.124122 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:49:33.125235 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:49:33.125721 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:49:33.125855 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:49:33.128323 (+ 2468us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:49:33.128328 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:49:33.128328 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:49:33.128346 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:49:33.130252 (+ 1906us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:49:33.130725 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:49:33.130728 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:49:33.130728 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:49:33.130831 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:49:33.131099 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:49:33.131100 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:49:33.131100 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:49:33.131332 (+ 232us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:49:33.131425 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0806 03:49:33.131729 (+ 304us) server_negotiation.cc:299] Negotiation successful
0806 03:49:33.131862 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2784,"thread_start_us":82,"threads_started":1}
I20260806 03:49:33.133678 571 heartbeater.cc:347] Connected to a master server at 127.11.253.254:44939
I20260806 03:49:33.133793 571 heartbeater.cc:464] Registering TS with master...
I20260806 03:49:33.133999 571 heartbeater.cc:511] Master 127.11.253.254:44939 requested a full tablet report, sending...
I20260806 03:49:33.134791 32761 ts_manager.cc:194] Registered new tserver with Master: 67abaec1b40248afb1a33088431ae1d5 (127.11.253.193:36851)
I20260806 03:49:33.135982 32761 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.253.193@KRBTEST.COM'} at 127.11.253.193:46651
I20260806 03:49:33.137131 571 heartbeater.cc:503] Master 127.11.253.254:44939 was elected leader, sending a full tablet report...
Aug 06 03:49:33 dist-test-slave-6ntk krb5kdc[31917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988172, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.253.254@KRBTEST.COM for kudu/127.11.253.193@KRBTEST.COM
I20260806 03:49:33.143591 580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:49:33.135604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48768 (local address 127.11.253.193:36851)
0806 03:49:33.135730 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0806 03:49:33.135734 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 03:49:33.135743 (+ 9us) server_negotiation.cc:407] Connection header received
0806 03:49:33.135774 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:49:33.135775 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:49:33.135791 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:49:33.135876 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:49:33.136363 (+ 487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:49:33.137055 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:49:33.138073 (+ 1018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:49:33.138377 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:49:33.138458 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:49:33.140633 (+ 2175us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:49:33.140641 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:49:33.140641 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:49:33.140648 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:49:33.142278 (+ 1630us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:49:33.142697 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:49:33.142700 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:49:33.142701 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:49:33.142761 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:49:33.142975 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:49:33.142976 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:49:33.142976 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:49:33.143118 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:49:33.143213 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0806 03:49:33.143404 (+ 191us) server_negotiation.cc:299] Negotiation successful
0806 03:49:33.143499 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102,"thread_start_us":62,"threads_started":1}
I20260806 03:49:33.143591 579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:49:33.135372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.193:36851 (local address 127.0.0.1:48768)
0806 03:49:33.135545 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0806 03:49:33.135591 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0806 03:49:33.135661 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:49:33.135891 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:49:33.135895 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:49:33.136235 (+ 340us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:49:33.136295 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:49:33.136297 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:49:33.137152 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:49:33.137153 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:49:33.137962 (+ 809us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:49:33.137965 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:49:33.138458 (+ 493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:49:33.138460 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:49:33.138539 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:49:33.139030 (+ 491us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:49:33.139035 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:49:33.140539 (+ 1504us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:49:33.142382 (+ 1843us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:49:33.142389 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:49:33.142391 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:49:33.142595 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:49:33.142836 (+ 241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:49:33.142837 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:49:33.142837 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:49:33.142900 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:49:33.143193 (+ 293us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:49:33.143200 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:49:33.143253 (+ 53us) client_negotiation.cc:769] Sending connection context
0806 03:49:33.143364 (+ 111us) client_negotiation.cc:240] Negotiation successful
0806 03:49:33.143489 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":82,"threads_started":1}
I20260806 03:49:33.144095 473 tablet_service.cc:1514] Processing DeleteTablet for tablet 1dfe4e16c27e467a88e3aa7a83e27f98 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 03:49:32 UTC) from {username='kudu', principal='kudu/127.11.253.254@KRBTEST.COM'} at 127.0.0.1:48768
I20260806 03:49:33.144626 582 tablet_replica.cc:330] stopping tablet replica
I20260806 03:49:33.144747 582 raft_consensus.cc:2229] T 1dfe4e16c27e467a88e3aa7a83e27f98 P 67abaec1b40248afb1a33088431ae1d5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 03:49:33.144846 582 raft_consensus.cc:2258] T 1dfe4e16c27e467a88e3aa7a83e27f98 P 67abaec1b40248afb1a33088431ae1d5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 03:49:33.145164 582 ts_tablet_manager.cc:1905] T 1dfe4e16c27e467a88e3aa7a83e27f98 P 67abaec1b40248afb1a33088431ae1d5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 03:49:33.147325 582 ts_tablet_manager.cc:1918] T 1dfe4e16c27e467a88e3aa7a83e27f98 P 67abaec1b40248afb1a33088431ae1d5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 03:49:33.147421 582 log.cc:1196] T 1dfe4e16c27e467a88e3aa7a83e27f98 P 67abaec1b40248afb1a33088431ae1d5: Deleting WAL directory at /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/minicluster-data/ts-0/wal/wals/1dfe4e16c27e467a88e3aa7a83e27f98
I20260806 03:49:33.147666 582 ts_tablet_manager.cc:1939] T 1dfe4e16c27e467a88e3aa7a83e27f98 P 67abaec1b40248afb1a33088431ae1d5: Deleting consensus metadata
I20260806 03:49:33.148247 32743 catalog_manager.cc:4841] TS 67abaec1b40248afb1a33088431ae1d5 (127.11.253.193:36851): tablet 1dfe4e16c27e467a88e3aa7a83e27f98 (table test-table [id=e6aa44c02cab46b98bb5361fafa62512]) successfully deleted
W20260806 03:49:33.212781 576 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:49:33.213016 576 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:49:33.213042 576 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:49:33.214330 576 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:49:33.214380 576 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:49:33.214409 576 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.253.194
I20260806 03:49:33.215924 576 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.253.212:40573
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.194.keytab
--principal=kudu/127.11.253.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.253.194:37941
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.253.194
--webserver_port=39759
--tserver_master_addrs=127.11.253.254:44939
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.576
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.253.194
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:49:33.216190 576 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:49:33.216456 576 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:49:33.221673 588 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:49:33.221683 589 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:49:33.221683 593 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:49:33.221972 576 server_base.cc:1034] running on GCE node
I20260806 03:49:33.222205 576 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:49:33.222460 576 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:49:33.223640 576 hybrid_clock.cc:648] HybridClock initialized: now 1785988173223628 us; error 31 us; skew 500 ppm
Aug 06 03:49:33 dist-test-slave-6ntk krb5kdc[31917](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988173, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.253.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:49:33.226353 576 init.cc:377] Logged in from keytab as kudu/127.11.253.194@KRBTEST.COM (short username kudu)
I20260806 03:49:33.226832 576 webserver.cc:460] Webserver started at http://127.11.253.194:39759/ using document root <none> and password file <none>
I20260806 03:49:33.227022 576 fs_manager.cc:359] Metadata directory not provided
I20260806 03:49:33.227075 576 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:49:33.228374 576 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 03:49:33.228935 602 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:49:33.229118 576 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 03:49:33.229211 576 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/minicluster-data/ts-1/data,/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/minicluster-data/ts-1/wal
uuid: "df657246a4be41cdb398d7c7e988d8e4"
format_stamp: "Formatted at 2026-08-06 03:49:32 on dist-test-slave-6ntk"
server_key: "b713a4254580841280323bb428a9e14f"
server_key_iv: "16bdd544f6ea35651e971f1ea83e3d4e"
server_key_version: "encryptionkey@0"
I20260806 03:49:33.229638 576 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:49:33.243278 576 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:49:33.245543 576 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:49:33.245653 576 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:49:33.245831 576 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 03:49:33.246198 576 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 03:49:33.246233 576 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 03:49:33.246255 576 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 03:49:33.246268 576 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 03:49:33.252068 576 rpc_server.cc:307] RPC server started. Bound to: 127.11.253.194:37941
I20260806 03:49:33.252153 755 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.253.194:37941 every 8 connection(s)
I20260806 03:49:33.252493 576 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/minicluster-data/ts-1/data/info.pb
Aug 06 03:49:33 dist-test-slave-6ntk krb5kdc[31917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988173, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.253.194@KRBTEST.COM for kudu/127.11.253.254@KRBTEST.COM
I20260806 03:49:33.259953 12279 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskToVWRe/build/release/bin/kudu as pid 576
I20260806 03:49:33.260227 12279 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskToVWRe/build/release/bin/kudu
/tmp/dist-test-taskToVWRe/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.253.195:39103
--local_ip_for_outbound_sockets=127.11.253.195
--tserver_master_addrs=127.11.253.254:44939
--webserver_port=43749
--webserver_interface=127.11.253.195
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.195.keytab
--principal=kudu/127.11.253.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.253.212:40573
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 03:49:33.261889 575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:49:33.253232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.253.194:42719 (local address 127.11.253.254:44939)
0806 03:49:33.253303 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0806 03:49:33.253306 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 03:49:33.253553 (+ 247us) server_negotiation.cc:407] Connection header received
0806 03:49:33.254100 (+ 547us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:49:33.254101 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:49:33.254111 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:49:33.254165 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:49:33.255129 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:49:33.255638 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:49:33.256600 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:49:33.256811 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:49:33.256869 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:49:33.258877 (+ 2008us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:49:33.258879 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:49:33.258879 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:49:33.258885 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:49:33.260424 (+ 1539us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:49:33.260854 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:49:33.260855 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:49:33.260856 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:49:33.260921 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:49:33.261170 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:49:33.261170 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:49:33.261171 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:49:33.261298 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:49:33.261382 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0806 03:49:33.261646 (+ 264us) server_negotiation.cc:299] Negotiation successful
0806 03:49:33.261745 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260806 03:49:33.262172 758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:49:33.253227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:44939 (local address 127.11.253.194:42719)
0806 03:49:33.253427 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0806 03:49:33.253463 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0806 03:49:33.254008 (+ 545us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:49:33.254228 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:49:33.254232 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:49:33.254805 (+ 573us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:49:33.255046 (+ 241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:49:33.255051 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:49:33.255723 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:49:33.255724 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:49:33.256524 (+ 800us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:49:33.256526 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:49:33.256885 (+ 359us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:49:33.256886 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:49:33.256979 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:49:33.257415 (+ 436us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:49:33.257423 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:49:33.258783 (+ 1360us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:49:33.260544 (+ 1761us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:49:33.260551 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:49:33.260553 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:49:33.260775 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:49:33.260990 (+ 215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:49:33.260991 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:49:33.260992 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:49:33.261094 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:49:33.261372 (+ 278us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:49:33.261380 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:49:33.261507 (+ 127us) client_negotiation.cc:769] Sending connection context
0806 03:49:33.261611 (+ 104us) client_negotiation.cc:240] Negotiation successful
0806 03:49:33.261768 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":96,"threads_started":1}
I20260806 03:49:33.262655 756 heartbeater.cc:347] Connected to a master server at 127.11.253.254:44939
I20260806 03:49:33.262748 756 heartbeater.cc:464] Registering TS with master...
I20260806 03:49:33.262948 756 heartbeater.cc:511] Master 127.11.253.254:44939 requested a full tablet report, sending...
I20260806 03:49:33.263321 32761 ts_manager.cc:194] Registered new tserver with Master: df657246a4be41cdb398d7c7e988d8e4 (127.11.253.194:37941)
I20260806 03:49:33.263851 32761 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.253.194@KRBTEST.COM'} at 127.11.253.194:42719
W20260806 03:49:33.346367 761 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:49:33.346580 761 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:49:33.346602 761 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:49:33.347682 761 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:49:33.347718 761 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:49:33.347743 761 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.253.195
I20260806 03:49:33.349128 761 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.253.212:40573
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.195.keytab
--principal=kudu/127.11.253.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.253.195:39103
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.253.195
--webserver_port=43749
--tserver_master_addrs=127.11.253.254:44939
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.761
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.253.195
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:49:33.349327 761 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:49:33.349526 761 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:49:33.354700 769 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:49:33.354697 768 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:49:33.354697 773 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:49:33.354974 761 server_base.cc:1034] running on GCE node
I20260806 03:49:33.355217 761 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:49:33.355468 761 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:49:33.356642 761 hybrid_clock.cc:648] HybridClock initialized: now 1785988173356633 us; error 29 us; skew 500 ppm
Aug 06 03:49:33 dist-test-slave-6ntk krb5kdc[31917](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988173, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.253.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:49:33.359355 761 init.cc:377] Logged in from keytab as kudu/127.11.253.195@KRBTEST.COM (short username kudu)
I20260806 03:49:33.359809 761 webserver.cc:460] Webserver started at http://127.11.253.195:43749/ using document root <none> and password file <none>
I20260806 03:49:33.360003 761 fs_manager.cc:359] Metadata directory not provided
I20260806 03:49:33.360055 761 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:49:33.361328 761 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 03:49:33.361979 785 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:49:33.362166 761 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260806 03:49:33.362268 761 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/minicluster-data/ts-2/data,/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/minicluster-data/ts-2/wal
uuid: "403aacadd2b546c1b7289fad1f6066eb"
format_stamp: "Formatted at 2026-08-06 03:49:32 on dist-test-slave-6ntk"
server_key: "4d1b136f6886e0b91ec96dacee8cc8da"
server_key_iv: "66f109e8c54a9e4fbe7df92edd2efe14"
server_key_version: "encryptionkey@0"
I20260806 03:49:33.362592 761 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:49:33.377199 761 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:49:33.379475 761 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:49:33.379592 761 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:49:33.379787 761 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 03:49:33.380106 761 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 03:49:33.380141 761 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 03:49:33.380162 761 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 03:49:33.380213 761 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 03:49:33.385915 761 rpc_server.cc:307] RPC server started. Bound to: 127.11.253.195:39103
I20260806 03:49:33.385994 942 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.253.195:39103 every 8 connection(s)
I20260806 03:49:33.386363 761 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/minicluster-data/ts-2/data/info.pb
I20260806 03:49:33.386911 12279 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskToVWRe/build/release/bin/kudu as pid 761
Aug 06 03:49:33 dist-test-slave-6ntk krb5kdc[31917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988173, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.253.195@KRBTEST.COM for kudu/127.11.253.254@KRBTEST.COM
I20260806 03:49:33.395419 947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:49:33.387324 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36812 (local address 127.11.253.254:44939)
0806 03:49:33.387448 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0806 03:49:33.387450 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:49:33.387643 (+ 193us) server_negotiation.cc:407] Connection header received
0806 03:49:33.387689 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:49:33.387690 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:49:33.387704 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:49:33.387779 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:49:33.388543 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:49:33.389381 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:49:33.390552 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:49:33.390855 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:49:33.390974 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:49:33.392411 (+ 1437us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:49:33.392412 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:49:33.392412 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:49:33.392419 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:49:33.394095 (+ 1676us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:49:33.394491 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:49:33.394492 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:49:33.394493 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:49:33.394559 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:49:33.394788 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:49:33.394789 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:49:33.394790 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:49:33.394934 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:49:33.395153 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0806 03:49:33.395202 (+ 49us) server_negotiation.cc:299] Negotiation successful
0806 03:49:33.395296 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102,"thread_start_us":75,"threads_started":1}
I20260806 03:49:33.397221 575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:49:33.387077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.253.195:51187 (local address 127.11.253.254:44939)
0806 03:49:33.387130 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0806 03:49:33.387133 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 03:49:33.387414 (+ 281us) server_negotiation.cc:407] Connection header received
0806 03:49:33.388102 (+ 688us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:49:33.388103 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:49:33.388115 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:49:33.388188 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:49:33.389382 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:49:33.390214 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:49:33.391551 (+ 1337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:49:33.391867 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:49:33.391947 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:49:33.394301 (+ 2354us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:49:33.394302 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:49:33.394302 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:49:33.394308 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:49:33.395961 (+ 1653us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:49:33.396412 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:49:33.396413 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:49:33.396413 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:49:33.396462 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:49:33.396629 (+ 167us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:49:33.396629 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:49:33.396629 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:49:33.396708 (+ 79us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:49:33.396753 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0806 03:49:33.397100 (+ 347us) server_negotiation.cc:299] Negotiation successful
0806 03:49:33.397162 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260806 03:49:33.397555 945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:49:33.387070 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:44939 (local address 127.11.253.195:51187)
0806 03:49:33.387257 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0806 03:49:33.387311 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0806 03:49:33.387990 (+ 679us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:49:33.388238 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:49:33.388241 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:49:33.389006 (+ 765us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:49:33.389281 (+ 275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:49:33.389287 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:49:33.390300 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:49:33.390301 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:49:33.391464 (+ 1163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:49:33.391468 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:49:33.391954 (+ 486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:49:33.391955 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:49:33.392056 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:49:33.392530 (+ 474us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:49:33.392538 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:49:33.394208 (+ 1670us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:49:33.396036 (+ 1828us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:49:33.396042 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:49:33.396044 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:49:33.396350 (+ 306us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:49:33.396505 (+ 155us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:49:33.396505 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:49:33.396506 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:49:33.396594 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:49:33.396798 (+ 204us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:49:33.396805 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:49:33.396954 (+ 149us) client_negotiation.cc:769] Sending connection context
0806 03:49:33.397071 (+ 117us) client_negotiation.cc:240] Negotiation successful
0806 03:49:33.397271 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":93,"threads_started":1}
I20260806 03:49:33.398100 943 heartbeater.cc:347] Connected to a master server at 127.11.253.254:44939
I20260806 03:49:33.398186 943 heartbeater.cc:464] Registering TS with master...
I20260806 03:49:33.398415 943 heartbeater.cc:511] Master 127.11.253.254:44939 requested a full tablet report, sending...
I20260806 03:49:33.398793 32761 ts_manager.cc:194] Registered new tserver with Master: 403aacadd2b546c1b7289fad1f6066eb (127.11.253.195:39103)
I20260806 03:49:33.399225 32761 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.253.195@KRBTEST.COM'} at 127.11.253.195:51187
I20260806 03:49:33.407164 12279 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 03:49:33.410256 575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:49:33.407564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36820 (local address 127.11.253.254:44939)
0806 03:49:33.407674 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0806 03:49:33.407676 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:49:33.407793 (+ 117us) server_negotiation.cc:407] Connection header received
0806 03:49:33.407910 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:49:33.407910 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:49:33.407958 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:49:33.408047 (+ 89us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 03:49:33.408183 (+ 136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:49:33.408654 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:49:33.409441 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:49:33.409645 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:49:33.409719 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:49:33.409933 (+ 214us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 03:49:33.410041 (+ 108us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 03:49:33.410139 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0806 03:49:33.410159 (+ 20us) server_negotiation.cc:299] Negotiation successful
0806 03:49:33.410176 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260806 03:49:33.410563 32761 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:36820:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 03:49:33.412533 32761 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 03:49:33.415112 473 tablet_service.cc:1467] Processing CreateTablet for tablet 24a91b281a7b488f8a1a6cd87e0b850c (DEFAULT_TABLE table=test-table [id=4c20b5c7a75249739902f2fa5126e1fd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 03:49:33.415330 473 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 24a91b281a7b488f8a1a6cd87e0b850c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:49:33.416688 416 tablet_bootstrap.cc:492] T 24a91b281a7b488f8a1a6cd87e0b850c P 67abaec1b40248afb1a33088431ae1d5: Bootstrap starting.
I20260806 03:49:33.417138 416 tablet_bootstrap.cc:654] T 24a91b281a7b488f8a1a6cd87e0b850c P 67abaec1b40248afb1a33088431ae1d5: Neither blocks nor log segments found. Creating new log.
I20260806 03:49:33.418272 416 tablet_bootstrap.cc:492] T 24a91b281a7b488f8a1a6cd87e0b850c P 67abaec1b40248afb1a33088431ae1d5: No bootstrap required, opened a new log
I20260806 03:49:33.418368 416 ts_tablet_manager.cc:1397] T 24a91b281a7b488f8a1a6cd87e0b850c P 67abaec1b40248afb1a33088431ae1d5: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260806 03:49:33.418545 416 raft_consensus.cc:348] T 24a91b281a7b488f8a1a6cd87e0b850c P 67abaec1b40248afb1a33088431ae1d5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "403aacadd2b546c1b7289fad1f6066eb" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 39103 } } peers { permanent_uuid: "df657246a4be41cdb398d7c7e988d8e4" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 37941 } } peers { permanent_uuid: "67abaec1b40248afb1a33088431ae1d5" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 36851 } }
I20260806 03:49:33.418648 416 raft_consensus.cc:374] T 24a91b281a7b488f8a1a6cd87e0b850c P 67abaec1b40248afb1a33088431ae1d5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:49:33.418716 416 raft_consensus.cc:729] T 24a91b281a7b488f8a1a6cd87e0b850c P 67abaec1b40248afb1a33088431ae1d5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 67abaec1b40248afb1a33088431ae1d5, State: Initialized, Role: FOLLOWER
I20260806 03:49:33.418778 416 consensus_queue.cc:260] T 24a91b281a7b488f8a1a6cd87e0b850c P 67abaec1b40248afb1a33088431ae1d5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "403aacadd2b546c1b7289fad1f6066eb" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 39103 } } peers { permanent_uuid: "df657246a4be41cdb398d7c7e988d8e4" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 37941 } } peers { permanent_uuid: "67abaec1b40248afb1a33088431ae1d5" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 36851 } }
I20260806 03:49:33.418898 416 ts_tablet_manager.cc:1428] T 24a91b281a7b488f8a1a6cd87e0b850c P 67abaec1b40248afb1a33088431ae1d5: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260806 03:49:33.419065 955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:49:33.414891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.194:37941 (local address 127.0.0.1:51998)
0806 03:49:33.415016 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0806 03:49:33.415029 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 03:49:33.415093 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:49:33.415551 (+ 458us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:49:33.415553 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:49:33.415555 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 03:49:33.415600 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:49:33.415601 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:49:33.416592 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:49:33.416593 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:49:33.417798 (+ 1205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:49:33.417801 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:49:33.418814 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:49:33.418815 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:49:33.418888 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:49:33.418909 (+ 21us) client_negotiation.cc:769] Sending connection context
0806 03:49:33.418962 (+ 53us) client_negotiation.cc:240] Negotiation successful
0806 03:49:33.419002 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":67,"threads_started":1}
I20260806 03:49:33.419396 957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:49:33.415092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51998 (local address 127.11.253.194:37941)
0806 03:49:33.415324 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0806 03:49:33.415329 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 03:49:33.415342 (+ 13us) server_negotiation.cc:407] Connection header received
0806 03:49:33.415392 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:49:33.415394 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:49:33.415414 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:49:33.415519 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 03:49:33.415689 (+ 170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:49:33.416472 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:49:33.417926 (+ 1454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:49:33.418628 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:49:33.418720 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:49:33.418777 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0806 03:49:33.419256 (+ 479us) server_negotiation.cc:299] Negotiation successful
0806 03:49:33.419305 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":184,"thread_start_us":103,"threads_started":1}
I20260806 03:49:33.420114 666 tablet_service.cc:1467] Processing CreateTablet for tablet 24a91b281a7b488f8a1a6cd87e0b850c (DEFAULT_TABLE table=test-table [id=4c20b5c7a75249739902f2fa5126e1fd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 03:49:33.420435 666 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 24a91b281a7b488f8a1a6cd87e0b850c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:49:33.420830 579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:49:33.414791 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.195:39103 (local address 127.0.0.1:40016)
0806 03:49:33.414825 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0806 03:49:33.414836 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 03:49:33.414886 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:49:33.417019 (+ 2133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:49:33.417021 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:49:33.417023 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 03:49:33.417087 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:49:33.417089 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:49:33.418342 (+ 1253us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:49:33.418343 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:49:33.419547 (+ 1204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:49:33.419550 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:49:33.420554 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:49:33.420555 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:49:33.420636 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:49:33.420641 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 03:49:33.420711 (+ 70us) client_negotiation.cc:240] Negotiation successful
0806 03:49:33.420759 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17}
I20260806 03:49:33.420863 956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:49:33.414891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40016 (local address 127.11.253.195:39103)
0806 03:49:33.416807 (+ 1916us) server_negotiation.cc:206] Beginning negotiation
0806 03:49:33.416812 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 03:49:33.416822 (+ 10us) server_negotiation.cc:407] Connection header received
0806 03:49:33.416869 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:49:33.416871 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:49:33.416887 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:49:33.417001 (+ 114us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 03:49:33.417183 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:49:33.418226 (+ 1043us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:49:33.419731 (+ 1505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:49:33.420429 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:49:33.420534 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:49:33.420594 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0806 03:49:33.420732 (+ 138us) server_negotiation.cc:299] Negotiation successful
0806 03:49:33.420775 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1882,"thread_start_us":122,"threads_started":1}
I20260806 03:49:33.421592 854 tablet_service.cc:1467] Processing CreateTablet for tablet 24a91b281a7b488f8a1a6cd87e0b850c (DEFAULT_TABLE table=test-table [id=4c20b5c7a75249739902f2fa5126e1fd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 03:49:33.421959 854 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 24a91b281a7b488f8a1a6cd87e0b850c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:49:33.424041 959 tablet_bootstrap.cc:492] T 24a91b281a7b488f8a1a6cd87e0b850c P df657246a4be41cdb398d7c7e988d8e4: Bootstrap starting.
I20260806 03:49:33.424626 959 tablet_bootstrap.cc:654] T 24a91b281a7b488f8a1a6cd87e0b850c P df657246a4be41cdb398d7c7e988d8e4: Neither blocks nor log segments found. Creating new log.
I20260806 03:49:33.425132 959 log.cc:824] T 24a91b281a7b488f8a1a6cd87e0b850c P df657246a4be41cdb398d7c7e988d8e4: Log is configured to *not* fsync() on all Append() calls
I20260806 03:49:33.425212 960 tablet_bootstrap.cc:492] T 24a91b281a7b488f8a1a6cd87e0b850c P 403aacadd2b546c1b7289fad1f6066eb: Bootstrap starting.
I20260806 03:49:33.425765 960 tablet_bootstrap.cc:654] T 24a91b281a7b488f8a1a6cd87e0b850c P 403aacadd2b546c1b7289fad1f6066eb: Neither blocks nor log segments found. Creating new log.
I20260806 03:49:33.425978 959 tablet_bootstrap.cc:492] T 24a91b281a7b488f8a1a6cd87e0b850c P df657246a4be41cdb398d7c7e988d8e4: No bootstrap required, opened a new log
I20260806 03:49:33.426059 959 ts_tablet_manager.cc:1397] T 24a91b281a7b488f8a1a6cd87e0b850c P df657246a4be41cdb398d7c7e988d8e4: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 03:49:33.426299 960 log.cc:824] T 24a91b281a7b488f8a1a6cd87e0b850c P 403aacadd2b546c1b7289fad1f6066eb: Log is configured to *not* fsync() on all Append() calls
I20260806 03:49:33.427026 960 tablet_bootstrap.cc:492] T 24a91b281a7b488f8a1a6cd87e0b850c P 403aacadd2b546c1b7289fad1f6066eb: No bootstrap required, opened a new log
I20260806 03:49:33.427109 960 ts_tablet_manager.cc:1397] T 24a91b281a7b488f8a1a6cd87e0b850c P 403aacadd2b546c1b7289fad1f6066eb: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 03:49:33.427703 959 raft_consensus.cc:348] T 24a91b281a7b488f8a1a6cd87e0b850c P df657246a4be41cdb398d7c7e988d8e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "403aacadd2b546c1b7289fad1f6066eb" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 39103 } } peers { permanent_uuid: "df657246a4be41cdb398d7c7e988d8e4" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 37941 } } peers { permanent_uuid: "67abaec1b40248afb1a33088431ae1d5" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 36851 } }
I20260806 03:49:33.427850 959 raft_consensus.cc:374] T 24a91b281a7b488f8a1a6cd87e0b850c P df657246a4be41cdb398d7c7e988d8e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:49:33.427896 959 raft_consensus.cc:729] T 24a91b281a7b488f8a1a6cd87e0b850c P df657246a4be41cdb398d7c7e988d8e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: df657246a4be41cdb398d7c7e988d8e4, State: Initialized, Role: FOLLOWER
I20260806 03:49:33.428102 959 consensus_queue.cc:260] T 24a91b281a7b488f8a1a6cd87e0b850c P df657246a4be41cdb398d7c7e988d8e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "403aacadd2b546c1b7289fad1f6066eb" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 39103 } } peers { permanent_uuid: "df657246a4be41cdb398d7c7e988d8e4" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 37941 } } peers { permanent_uuid: "67abaec1b40248afb1a33088431ae1d5" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 36851 } }
I20260806 03:49:33.428472 959 ts_tablet_manager.cc:1428] T 24a91b281a7b488f8a1a6cd87e0b850c P df657246a4be41cdb398d7c7e988d8e4: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 03:49:33.428491 756 heartbeater.cc:503] Master 127.11.253.254:44939 was elected leader, sending a full tablet report...
I20260806 03:49:33.428860 960 raft_consensus.cc:348] T 24a91b281a7b488f8a1a6cd87e0b850c P 403aacadd2b546c1b7289fad1f6066eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "403aacadd2b546c1b7289fad1f6066eb" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 39103 } } peers { permanent_uuid: "df657246a4be41cdb398d7c7e988d8e4" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 37941 } } peers { permanent_uuid: "67abaec1b40248afb1a33088431ae1d5" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 36851 } }
I20260806 03:49:33.429001 960 raft_consensus.cc:374] T 24a91b281a7b488f8a1a6cd87e0b850c P 403aacadd2b546c1b7289fad1f6066eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:49:33.429044 960 raft_consensus.cc:729] T 24a91b281a7b488f8a1a6cd87e0b850c P 403aacadd2b546c1b7289fad1f6066eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 403aacadd2b546c1b7289fad1f6066eb, State: Initialized, Role: FOLLOWER
I20260806 03:49:33.429152 960 consensus_queue.cc:260] T 24a91b281a7b488f8a1a6cd87e0b850c P 403aacadd2b546c1b7289fad1f6066eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "403aacadd2b546c1b7289fad1f6066eb" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 39103 } } peers { permanent_uuid: "df657246a4be41cdb398d7c7e988d8e4" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 37941 } } peers { permanent_uuid: "67abaec1b40248afb1a33088431ae1d5" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 36851 } }
I20260806 03:49:33.429446 943 heartbeater.cc:503] Master 127.11.253.254:44939 was elected leader, sending a full tablet report...
I20260806 03:49:33.429428 960 ts_tablet_manager.cc:1428] T 24a91b281a7b488f8a1a6cd87e0b850c P 403aacadd2b546c1b7289fad1f6066eb: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
W20260806 03:49:33.503086 757 tablet.cc:2369] T 24a91b281a7b488f8a1a6cd87e0b850c P df657246a4be41cdb398d7c7e988d8e4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 03:49:33.598816 970 raft_consensus.cc:482] T 24a91b281a7b488f8a1a6cd87e0b850c P df657246a4be41cdb398d7c7e988d8e4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 03:49:33.599009 970 raft_consensus.cc:504] T 24a91b281a7b488f8a1a6cd87e0b850c P df657246a4be41cdb398d7c7e988d8e4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "403aacadd2b546c1b7289fad1f6066eb" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 39103 } } peers { permanent_uuid: "df657246a4be41cdb398d7c7e988d8e4" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 37941 } } peers { permanent_uuid: "67abaec1b40248afb1a33088431ae1d5" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 36851 } }
I20260806 03:49:33.599440 970 leader_election.cc:288] T 24a91b281a7b488f8a1a6cd87e0b850c P df657246a4be41cdb398d7c7e988d8e4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 403aacadd2b546c1b7289fad1f6066eb (127.11.253.195:39103), 67abaec1b40248afb1a33088431ae1d5 (127.11.253.193:36851)
I20260806 03:49:33.603439 956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:49:33.599738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.253.194:50061 (local address 127.11.253.195:39103)
0806 03:49:33.599793 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0806 03:49:33.599795 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:49:33.599948 (+ 153us) server_negotiation.cc:407] Connection header received
0806 03:49:33.600000 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:49:33.600002 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:49:33.600011 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:49:33.600082 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 03:49:33.600285 (+ 203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:49:33.600894 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:49:33.602481 (+ 1587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:49:33.603044 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:49:33.603125 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:49:33.603137 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0806 03:49:33.603325 (+ 188us) server_negotiation.cc:299] Negotiation successful
0806 03:49:33.603366 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260806 03:49:33.603440 580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:49:33.599732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.253.194:42193 (local address 127.11.253.193:36851)
0806 03:49:33.599774 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0806 03:49:33.599776 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:49:33.599787 (+ 11us) server_negotiation.cc:407] Connection header received
0806 03:49:33.599878 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:49:33.599880 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:49:33.599912 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:49:33.599984 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 03:49:33.600192 (+ 208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:49:33.600924 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:49:33.602481 (+ 1557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:49:33.603120 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:49:33.603195 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:49:33.603299 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0806 03:49:33.603323 (+ 24us) server_negotiation.cc:299] Negotiation successful
0806 03:49:33.603367 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260806 03:49:33.603446 758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:49:33.599665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.193:36851 (local address 127.11.253.194:42193)
0806 03:49:33.599722 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0806 03:49:33.599734 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 03:49:33.599781 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:49:33.600011 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:49:33.600013 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:49:33.600015 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 03:49:33.600096 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:49:33.600098 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:49:33.601012 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:49:33.601013 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:49:33.602381 (+ 1368us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:49:33.602384 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:49:33.603194 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:49:33.603194 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:49:33.603264 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:49:33.603267 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 03:49:33.603309 (+ 42us) client_negotiation.cc:240] Negotiation successful
0806 03:49:33.603357 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38,"mutex_wait_us":110}
I20260806 03:49:33.603453 972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:49:33.599686 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.195:39103 (local address 127.11.253.194:50061)
0806 03:49:33.599871 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0806 03:49:33.599881 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 03:49:33.599940 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:49:33.600094 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:49:33.600095 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:49:33.600096 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 03:49:33.600187 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:49:33.600189 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:49:33.601012 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:49:33.601013 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:49:33.602381 (+ 1368us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:49:33.602384 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:49:33.603130 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:49:33.603130 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:49:33.603220 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:49:33.603224 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 03:49:33.603299 (+ 75us) client_negotiation.cc:240] Negotiation successful
0806 03:49:33.603348 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":168,"mutex_wait_us":51,"thread_start_us":117,"threads_started":1}
I20260806 03:49:33.603762 500 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "24a91b281a7b488f8a1a6cd87e0b850c" candidate_uuid: "df657246a4be41cdb398d7c7e988d8e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "67abaec1b40248afb1a33088431ae1d5" is_pre_election: true
I20260806 03:49:33.603834 879 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "24a91b281a7b488f8a1a6cd87e0b850c" candidate_uuid: "df657246a4be41cdb398d7c7e988d8e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "403aacadd2b546c1b7289fad1f6066eb" is_pre_election: true
I20260806 03:49:33.603968 879 raft_consensus.cc:2454] T 24a91b281a7b488f8a1a6cd87e0b850c P 403aacadd2b546c1b7289fad1f6066eb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate df657246a4be41cdb398d7c7e988d8e4 in term 0.
I20260806 03:49:33.603962 500 raft_consensus.cc:2454] T 24a91b281a7b488f8a1a6cd87e0b850c P 67abaec1b40248afb1a33088431ae1d5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate df657246a4be41cdb398d7c7e988d8e4 in term 0.
I20260806 03:49:33.604187 606 leader_election.cc:302] T 24a91b281a7b488f8a1a6cd87e0b850c P df657246a4be41cdb398d7c7e988d8e4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 67abaec1b40248afb1a33088431ae1d5, df657246a4be41cdb398d7c7e988d8e4; no voters:
I20260806 03:49:33.604308 970 raft_consensus.cc:2781] T 24a91b281a7b488f8a1a6cd87e0b850c P df657246a4be41cdb398d7c7e988d8e4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 03:49:33.604365 970 raft_consensus.cc:482] T 24a91b281a7b488f8a1a6cd87e0b850c P df657246a4be41cdb398d7c7e988d8e4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 03:49:33.604388 970 raft_consensus.cc:3037] T 24a91b281a7b488f8a1a6cd87e0b850c P df657246a4be41cdb398d7c7e988d8e4 [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:49:33.605146 970 raft_consensus.cc:504] T 24a91b281a7b488f8a1a6cd87e0b850c P df657246a4be41cdb398d7c7e988d8e4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "403aacadd2b546c1b7289fad1f6066eb" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 39103 } } peers { permanent_uuid: "df657246a4be41cdb398d7c7e988d8e4" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 37941 } } peers { permanent_uuid: "67abaec1b40248afb1a33088431ae1d5" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 36851 } }
I20260806 03:49:33.605283 970 leader_election.cc:288] T 24a91b281a7b488f8a1a6cd87e0b850c P df657246a4be41cdb398d7c7e988d8e4 [CANDIDATE]: Term 1 election: Requested vote from peers 403aacadd2b546c1b7289fad1f6066eb (127.11.253.195:39103), 67abaec1b40248afb1a33088431ae1d5 (127.11.253.193:36851)
I20260806 03:49:33.605448 500 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "24a91b281a7b488f8a1a6cd87e0b850c" candidate_uuid: "df657246a4be41cdb398d7c7e988d8e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "67abaec1b40248afb1a33088431ae1d5"
I20260806 03:49:33.605489 879 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "24a91b281a7b488f8a1a6cd87e0b850c" candidate_uuid: "df657246a4be41cdb398d7c7e988d8e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "403aacadd2b546c1b7289fad1f6066eb"
I20260806 03:49:33.605531 500 raft_consensus.cc:3037] T 24a91b281a7b488f8a1a6cd87e0b850c P 67abaec1b40248afb1a33088431ae1d5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:49:33.605553 879 raft_consensus.cc:3037] T 24a91b281a7b488f8a1a6cd87e0b850c P 403aacadd2b546c1b7289fad1f6066eb [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:49:33.606436 879 raft_consensus.cc:2454] T 24a91b281a7b488f8a1a6cd87e0b850c P 403aacadd2b546c1b7289fad1f6066eb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate df657246a4be41cdb398d7c7e988d8e4 in term 1.
I20260806 03:49:33.606436 500 raft_consensus.cc:2454] T 24a91b281a7b488f8a1a6cd87e0b850c P 67abaec1b40248afb1a33088431ae1d5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate df657246a4be41cdb398d7c7e988d8e4 in term 1.
I20260806 03:49:33.606657 606 leader_election.cc:302] T 24a91b281a7b488f8a1a6cd87e0b850c P df657246a4be41cdb398d7c7e988d8e4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 67abaec1b40248afb1a33088431ae1d5, df657246a4be41cdb398d7c7e988d8e4; no voters:
I20260806 03:49:33.606745 970 raft_consensus.cc:2781] T 24a91b281a7b488f8a1a6cd87e0b850c P df657246a4be41cdb398d7c7e988d8e4 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 03:49:33.606909 970 raft_consensus.cc:686] T 24a91b281a7b488f8a1a6cd87e0b850c P df657246a4be41cdb398d7c7e988d8e4 [term 1 LEADER]: Becoming Leader. State: Replica: df657246a4be41cdb398d7c7e988d8e4, State: Running, Role: LEADER
I20260806 03:49:33.607017 970 consensus_queue.cc:237] T 24a91b281a7b488f8a1a6cd87e0b850c P df657246a4be41cdb398d7c7e988d8e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "403aacadd2b546c1b7289fad1f6066eb" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 39103 } } peers { permanent_uuid: "df657246a4be41cdb398d7c7e988d8e4" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 37941 } } peers { permanent_uuid: "67abaec1b40248afb1a33088431ae1d5" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 36851 } }
I20260806 03:49:33.607751 32761 catalog_manager.cc:5495] T 24a91b281a7b488f8a1a6cd87e0b850c P df657246a4be41cdb398d7c7e988d8e4 reported cstate change: term changed from 0 to 1, leader changed from <none> to df657246a4be41cdb398d7c7e988d8e4 (127.11.253.194). New cstate: current_term: 1 leader_uuid: "df657246a4be41cdb398d7c7e988d8e4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "403aacadd2b546c1b7289fad1f6066eb" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 39103 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "df657246a4be41cdb398d7c7e988d8e4" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 37941 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "67abaec1b40248afb1a33088431ae1d5" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 36851 } health_report { overall_health: UNKNOWN } } }
W20260806 03:49:33.619159 572 tablet.cc:2369] T 24a91b281a7b488f8a1a6cd87e0b850c P 67abaec1b40248afb1a33088431ae1d5: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 03:49:33.636965 944 tablet.cc:2369] T 24a91b281a7b488f8a1a6cd87e0b850c P 403aacadd2b546c1b7289fad1f6066eb: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 03:49:33.650681 957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:49:33.648281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52010 (local address 127.11.253.194:37941)
0806 03:49:33.648337 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0806 03:49:33.648339 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:49:33.648390 (+ 51us) server_negotiation.cc:407] Connection header received
0806 03:49:33.648431 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:49:33.648432 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:49:33.648451 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:49:33.648504 (+ 53us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 03:49:33.648617 (+ 113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:49:33.649071 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:49:33.649837 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:49:33.650075 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:49:33.650130 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:49:33.650309 (+ 179us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 03:49:33.650416 (+ 107us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 03:49:33.650479 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0806 03:49:33.650566 (+ 87us) server_negotiation.cc:299] Negotiation successful
0806 03:49:33.650583 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260806 03:49:33.652709 879 raft_consensus.cc:1261] T 24a91b281a7b488f8a1a6cd87e0b850c P 403aacadd2b546c1b7289fad1f6066eb [term 1 FOLLOWER]: Refusing update from remote peer df657246a4be41cdb398d7c7e988d8e4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 03:49:33.652805 500 raft_consensus.cc:1261] T 24a91b281a7b488f8a1a6cd87e0b850c P 67abaec1b40248afb1a33088431ae1d5 [term 1 FOLLOWER]: Refusing update from remote peer df657246a4be41cdb398d7c7e988d8e4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 03:49:33.653074 970 consensus_queue.cc:1035] T 24a91b281a7b488f8a1a6cd87e0b850c P df657246a4be41cdb398d7c7e988d8e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "403aacadd2b546c1b7289fad1f6066eb" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 39103 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 03:49:33.653244 974 consensus_queue.cc:1035] T 24a91b281a7b488f8a1a6cd87e0b850c P df657246a4be41cdb398d7c7e988d8e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "67abaec1b40248afb1a33088431ae1d5" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 36851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 03:49:33.655570 981 mvcc.cc:205] Tried to move back new op lower bound from 7315407559278063616 to 7315407559094808576. Current Snapshot: MvccSnapshot[applied={T|T < 7315407559278063616}]
I20260806 03:49:33.656271 984 mvcc.cc:205] Tried to move back new op lower bound from 7315407559278063616 to 7315407559094808576. Current Snapshot: MvccSnapshot[applied={T|T < 7315407559278063616}]
I20260806 03:49:33.657764 32761 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:36820:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 03:49:33.657897 32761 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:36820:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 03:49:33.658774 32761 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9c6e110e6ae045d2992beccbc2b49594: Sending DeleteTablet for 3 replicas of tablet 24a91b281a7b488f8a1a6cd87e0b850c
I20260806 03:49:33.659116 854 tablet_service.cc:1514] Processing DeleteTablet for tablet 24a91b281a7b488f8a1a6cd87e0b850c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 03:49:33 UTC) from {username='kudu', principal='kudu/127.11.253.254@KRBTEST.COM'} at 127.0.0.1:40016
I20260806 03:49:33.659191 666 tablet_service.cc:1514] Processing DeleteTablet for tablet 24a91b281a7b488f8a1a6cd87e0b850c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 03:49:33 UTC) from {username='kudu', principal='kudu/127.11.253.254@KRBTEST.COM'} at 127.0.0.1:51998
I20260806 03:49:33.659231 473 tablet_service.cc:1514] Processing DeleteTablet for tablet 24a91b281a7b488f8a1a6cd87e0b850c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 03:49:33 UTC) from {username='kudu', principal='kudu/127.11.253.254@KRBTEST.COM'} at 127.0.0.1:48768
I20260806 03:49:33.659389 994 tablet_replica.cc:330] T 24a91b281a7b488f8a1a6cd87e0b850c P 403aacadd2b546c1b7289fad1f6066eb: stopping tablet replica
I20260806 03:49:33.659499 994 raft_consensus.cc:2229] T 24a91b281a7b488f8a1a6cd87e0b850c P 403aacadd2b546c1b7289fad1f6066eb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 03:49:33.659533 996 tablet_replica.cc:330] T 24a91b281a7b488f8a1a6cd87e0b850c P 67abaec1b40248afb1a33088431ae1d5: stopping tablet replica
I20260806 03:49:33.659533 12279 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskToVWRe/build/release/bin/kudu with pid 355
I20260806 03:49:33.659590 996 raft_consensus.cc:2229] T 24a91b281a7b488f8a1a6cd87e0b850c P 67abaec1b40248afb1a33088431ae1d5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 03:49:33.659628 996 raft_consensus.cc:2258] T 24a91b281a7b488f8a1a6cd87e0b850c P 67abaec1b40248afb1a33088431ae1d5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 03:49:33.659616 994 raft_consensus.cc:2258] T 24a91b281a7b488f8a1a6cd87e0b850c P 403aacadd2b546c1b7289fad1f6066eb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 03:49:33.665704 995 tablet_replica.cc:330] T 24a91b281a7b488f8a1a6cd87e0b850c P df657246a4be41cdb398d7c7e988d8e4: stopping tablet replica
I20260806 03:49:33.665901 995 raft_consensus.cc:2229] T 24a91b281a7b488f8a1a6cd87e0b850c P df657246a4be41cdb398d7c7e988d8e4 [term 1 LEADER]: Raft consensus shutting down.
I20260806 03:49:33.666090 995 raft_consensus.cc:2258] T 24a91b281a7b488f8a1a6cd87e0b850c P df657246a4be41cdb398d7c7e988d8e4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 03:49:33.667898 995 ts_tablet_manager.cc:1905] T 24a91b281a7b488f8a1a6cd87e0b850c P df657246a4be41cdb398d7c7e988d8e4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 03:49:33.669718 995 ts_tablet_manager.cc:1918] T 24a91b281a7b488f8a1a6cd87e0b850c P df657246a4be41cdb398d7c7e988d8e4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 03:49:33.669857 995 log.cc:1196] T 24a91b281a7b488f8a1a6cd87e0b850c P df657246a4be41cdb398d7c7e988d8e4: Deleting WAL directory at /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/minicluster-data/ts-1/wal/wals/24a91b281a7b488f8a1a6cd87e0b850c
I20260806 03:49:33.670163 995 ts_tablet_manager.cc:1939] T 24a91b281a7b488f8a1a6cd87e0b850c P df657246a4be41cdb398d7c7e988d8e4: Deleting consensus metadata
I20260806 03:49:33.670679 32741 catalog_manager.cc:4841] TS df657246a4be41cdb398d7c7e988d8e4 (127.11.253.194:37941): tablet 24a91b281a7b488f8a1a6cd87e0b850c (table test-table [id=4c20b5c7a75249739902f2fa5126e1fd]) successfully deleted
W20260806 03:49:33.673363 32743 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.253.193:36851 (error 108)
I20260806 03:49:33.673389 12279 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskToVWRe/build/release/bin/kudu with pid 576
I20260806 03:49:33.676434 994 ts_tablet_manager.cc:1905] T 24a91b281a7b488f8a1a6cd87e0b850c P 403aacadd2b546c1b7289fad1f6066eb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 03:49:33.678004 994 ts_tablet_manager.cc:1918] T 24a91b281a7b488f8a1a6cd87e0b850c P 403aacadd2b546c1b7289fad1f6066eb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 03:49:33.678081 994 log.cc:1196] T 24a91b281a7b488f8a1a6cd87e0b850c P 403aacadd2b546c1b7289fad1f6066eb: Deleting WAL directory at /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/minicluster-data/ts-2/wal/wals/24a91b281a7b488f8a1a6cd87e0b850c
I20260806 03:49:33.678352 994 ts_tablet_manager.cc:1939] T 24a91b281a7b488f8a1a6cd87e0b850c P 403aacadd2b546c1b7289fad1f6066eb: Deleting consensus metadata
I20260806 03:49:33.678857 32741 catalog_manager.cc:4841] TS 403aacadd2b546c1b7289fad1f6066eb (127.11.253.195:39103): tablet 24a91b281a7b488f8a1a6cd87e0b850c (table test-table [id=4c20b5c7a75249739902f2fa5126e1fd]) successfully deleted
I20260806 03:49:33.681396 579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:49:33.681220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.193:36851 (local address 127.0.0.1:48776)
0806 03:49:33.681311 (+ 91us) negotiation.cc:107] Waiting for socket to connect
0806 03:49:33.681364 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.253.193:36851: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
W20260806 03:49:33.681612 32743 catalog_manager.cc:4584] TS 67abaec1b40248afb1a33088431ae1d5 (127.11.253.193:36851): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 24a91b281a7b488f8a1a6cd87e0b850c: Network error: Client connection negotiation failed: client connection to 127.11.253.193:36851: connect: Connection refused (error 111)
I20260806 03:49:33.685549 12279 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskToVWRe/build/release/bin/kudu with pid 761
I20260806 03:49:33.694155 12279 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskToVWRe/build/release/bin/kudu with pid 32719
I20260806 03:49:33.703527 12279 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskToVWRe/build/release/bin/kudu
/tmp/dist-test-taskToVWRe/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.253.254:44939
--webserver_interface=127.11.253.254
--webserver_port=38907
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.254.keytab
--principal=kudu/127.11.253.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.253.212:40573
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.253.254:44939
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:49:33.788194 1000 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:49:33.788441 1000 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:49:33.788465 1000 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:49:33.789700 1000 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 03:49:33.789745 1000 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:49:33.789760 1000 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:49:33.789773 1000 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 03:49:33.789824 1000 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 03:49:33.791580 1000 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.253.212:40573
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.253.254:44939
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.254.keytab
--principal=kudu/127.11.253.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.253.254:44939
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.253.254
--webserver_port=38907
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1000
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:49:33.791805 1000 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:49:33.791991 1000 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:49:33.797161 1012 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:49:33.797166 1007 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:49:33.797425 1000 server_base.cc:1034] running on GCE node
W20260806 03:49:33.797163 1008 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:49:33.797771 1000 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:49:33.798135 1000 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:49:33.799288 1000 hybrid_clock.cc:648] HybridClock initialized: now 1785988173799266 us; error 36 us; skew 500 ppm
Aug 06 03:49:33 dist-test-slave-6ntk krb5kdc[31917](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988173, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.253.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:49:33.801847 1000 init.cc:377] Logged in from keytab as kudu/127.11.253.254@KRBTEST.COM (short username kudu)
I20260806 03:49:33.802424 1000 webserver.cc:460] Webserver started at http://127.11.253.254:38907/ using document root <none> and password file <none>
I20260806 03:49:33.802682 1000 fs_manager.cc:359] Metadata directory not provided
I20260806 03:49:33.802753 1000 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:49:33.804179 1000 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 03:49:33.804931 1023 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:49:33.805128 1000 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 03:49:33.805245 1000 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/minicluster-data/master-0/data,/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/minicluster-data/master-0/wal
uuid: "9c6e110e6ae045d2992beccbc2b49594"
format_stamp: "Formatted at 2026-08-06 03:49:31 on dist-test-slave-6ntk"
server_key: "c1b753a64abc5121b494e9ad05f3ce2c"
server_key_iv: "dbed1853eafb71a198456bd471eec18d"
server_key_version: "encryptionkey@0"
I20260806 03:49:33.805644 1000 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:49:33.830819 1000 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:49:33.833045 1000 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:49:33.833221 1000 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:49:33.837644 1000 rpc_server.cc:307] RPC server started. Bound to: 127.11.253.254:44939
I20260806 03:49:33.837687 1096 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.253.254:44939 every 8 connection(s)
I20260806 03:49:33.838157 1000 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/minicluster-data/master-0/data/info.pb
I20260806 03:49:33.839499 12279 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskToVWRe/build/release/bin/kudu as pid 1000
I20260806 03:49:33.839798 12279 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskToVWRe/build/release/bin/kudu
/tmp/dist-test-taskToVWRe/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.253.193:36851
--local_ip_for_outbound_sockets=127.11.253.193
--tserver_master_addrs=127.11.253.254:44939
--webserver_port=36067
--webserver_interface=127.11.253.193
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.193.keytab
--principal=kudu/127.11.253.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.253.212:40573
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 03:49:33.840029 1097 sys_catalog.cc:263] Verifying existing consensus state
I20260806 03:49:33.840711 1097 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9c6e110e6ae045d2992beccbc2b49594: Bootstrap starting.
I20260806 03:49:33.850298 1097 log.cc:824] T 00000000000000000000000000000000 P 9c6e110e6ae045d2992beccbc2b49594: Log is configured to *not* fsync() on all Append() calls
I20260806 03:49:33.852854 1097 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9c6e110e6ae045d2992beccbc2b49594: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 03:49:33.853173 1097 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9c6e110e6ae045d2992beccbc2b49594: Bootstrap complete.
I20260806 03:49:33.855705 1097 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9c6e110e6ae045d2992beccbc2b49594 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c6e110e6ae045d2992beccbc2b49594" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 44939 } }
I20260806 03:49:33.855937 1097 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9c6e110e6ae045d2992beccbc2b49594 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c6e110e6ae045d2992beccbc2b49594, State: Initialized, Role: FOLLOWER
I20260806 03:49:33.856113 1097 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9c6e110e6ae045d2992beccbc2b49594 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c6e110e6ae045d2992beccbc2b49594" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 44939 } }
I20260806 03:49:33.856189 1097 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9c6e110e6ae045d2992beccbc2b49594 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 03:49:33.856240 1097 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9c6e110e6ae045d2992beccbc2b49594 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 03:49:33.856297 1097 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9c6e110e6ae045d2992beccbc2b49594 [term 2 FOLLOWER]: Advancing to term 3
I20260806 03:49:33.857366 1097 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9c6e110e6ae045d2992beccbc2b49594 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c6e110e6ae045d2992beccbc2b49594" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 44939 } }
I20260806 03:49:33.857458 1097 leader_election.cc:302] T 00000000000000000000000000000000 P 9c6e110e6ae045d2992beccbc2b49594 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9c6e110e6ae045d2992beccbc2b49594; no voters:
I20260806 03:49:33.857703 1097 leader_election.cc:288] T 00000000000000000000000000000000 P 9c6e110e6ae045d2992beccbc2b49594 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260806 03:49:33.857741 1103 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9c6e110e6ae045d2992beccbc2b49594 [term 3 FOLLOWER]: Leader election won for term 3
I20260806 03:49:33.857944 1103 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9c6e110e6ae045d2992beccbc2b49594 [term 3 LEADER]: Becoming Leader. State: Replica: 9c6e110e6ae045d2992beccbc2b49594, State: Running, Role: LEADER
I20260806 03:49:33.858012 1097 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9c6e110e6ae045d2992beccbc2b49594 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 03:49:33.858134 1103 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9c6e110e6ae045d2992beccbc2b49594 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c6e110e6ae045d2992beccbc2b49594" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 44939 } }
I20260806 03:49:33.858907 1104 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9c6e110e6ae045d2992beccbc2b49594 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "9c6e110e6ae045d2992beccbc2b49594" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c6e110e6ae045d2992beccbc2b49594" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 44939 } } }
I20260806 03:49:33.859012 1104 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9c6e110e6ae045d2992beccbc2b49594 [sys.catalog]: This master's current role is: LEADER
I20260806 03:49:33.858925 1105 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9c6e110e6ae045d2992beccbc2b49594 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9c6e110e6ae045d2992beccbc2b49594. Latest consensus state: current_term: 3 leader_uuid: "9c6e110e6ae045d2992beccbc2b49594" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c6e110e6ae045d2992beccbc2b49594" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 44939 } } }
I20260806 03:49:33.859105 1105 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9c6e110e6ae045d2992beccbc2b49594 [sys.catalog]: This master's current role is: LEADER
I20260806 03:49:33.859295 1124 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 03:49:33.860009 1124 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 03:49:33.860144 1124 catalog_manager.cc:1259] Loaded cluster ID: b5c142d5173b48c8b1eaa358a27de72b
I20260806 03:49:33.860199 1124 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260806 03:49:33.860587 1124 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f11231189ad google::LogMessage::Fail() at ??:0
@ 0x7f112311b88a google::LogMessage::SendToLog() at ??:0
@ 0x7f11231184de google::LogMessage::Flush() at ??:0
@ 0x7f1123119a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f112b0390f1 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7f112b0428b9 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f1124fe60d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f1124fdb954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f1125e3f6db start_thread at ??:0
@ 0x7f1121c1661f clone at ??:0
W20260806 03:49:33.968462 1099 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:49:33.968773 1099 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:49:33.968815 1099 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:49:33.970582 1099 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:49:33.970649 1099 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:49:33.970692 1099 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.253.193
I20260806 03:49:33.972934 1099 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.253.212:40573
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.193.keytab
--principal=kudu/127.11.253.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.253.193:36851
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.253.193
--webserver_port=36067
--tserver_master_addrs=127.11.253.254:44939
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1099
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.253.193
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:49:33.973203 1099 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:49:33.973512 1099 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:49:33.979521 1155 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:49:33.979739 1160 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:49:33.979521 1154 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:49:33.980204 1099 server_base.cc:1034] running on GCE node
I20260806 03:49:33.980506 1099 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:49:33.980764 1099 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:49:33.981938 1099 hybrid_clock.cc:648] HybridClock initialized: now 1785988173981923 us; error 31 us; skew 500 ppm
Aug 06 03:49:33 dist-test-slave-6ntk krb5kdc[31917](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988173, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.253.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:49:33.984730 1099 init.cc:377] Logged in from keytab as kudu/127.11.253.193@KRBTEST.COM (short username kudu)
I20260806 03:49:33.985194 1099 webserver.cc:460] Webserver started at http://127.11.253.193:36067/ using document root <none> and password file <none>
I20260806 03:49:33.985432 1099 fs_manager.cc:359] Metadata directory not provided
I20260806 03:49:33.985504 1099 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:49:33.987087 1099 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 03:49:33.987879 1169 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:49:33.988078 1099 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 03:49:33.988190 1099 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/minicluster-data/ts-0/data,/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/minicluster-data/ts-0/wal
uuid: "67abaec1b40248afb1a33088431ae1d5"
format_stamp: "Formatted at 2026-08-06 03:49:32 on dist-test-slave-6ntk"
server_key: "97e4da2295ff60317613732a64c0f93b"
server_key_iv: "94348828464dd14f650c05dac0d9e775"
server_key_version: "encryptionkey@0"
I20260806 03:49:33.988569 1099 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:49:34.005252 1099 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:49:34.007934 1099 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:49:34.008117 1099 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:49:34.008375 1099 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 03:49:34.008862 1179 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260806 03:49:34.011149 1099 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260806 03:49:34.011217 1099 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.003s user 0.000s sys 0.000s
I20260806 03:49:34.011268 1099 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260806 03:49:34.011947 1099 ts_tablet_manager.cc:610] Registered 1 tablets
I20260806 03:49:34.012006 1099 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260806 03:49:34.012080 1179 tablet_bootstrap.cc:492] T 24a91b281a7b488f8a1a6cd87e0b850c P 67abaec1b40248afb1a33088431ae1d5: Bootstrap starting.
I20260806 03:49:34.019541 1099 rpc_server.cc:307] RPC server started. Bound to: 127.11.253.193:36851
I20260806 03:49:34.019697 1331 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.253.193:36851 every 8 connection(s)
I20260806 03:49:34.019985 1099 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/minicluster-data/ts-0/data/info.pb
I20260806 03:49:34.024430 1179 log.cc:824] T 24a91b281a7b488f8a1a6cd87e0b850c P 67abaec1b40248afb1a33088431ae1d5: Log is configured to *not* fsync() on all Append() calls
I20260806 03:49:34.026854 12279 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskToVWRe/build/release/bin/kudu as pid 1099
I20260806 03:49:34.027195 12279 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskToVWRe/build/release/bin/kudu
/tmp/dist-test-taskToVWRe/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.253.194:37941
--local_ip_for_outbound_sockets=127.11.253.194
--tserver_master_addrs=127.11.253.254:44939
--webserver_port=39759
--webserver_interface=127.11.253.194
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.194.keytab
--principal=kudu/127.11.253.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.253.212:40573
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 03:49:34.027590 1179 tablet_bootstrap.cc:492] T 24a91b281a7b488f8a1a6cd87e0b850c P 67abaec1b40248afb1a33088431ae1d5: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 03:49:34.027931 1179 tablet_bootstrap.cc:492] T 24a91b281a7b488f8a1a6cd87e0b850c P 67abaec1b40248afb1a33088431ae1d5: Bootstrap complete.
I20260806 03:49:34.029224 1179 ts_tablet_manager.cc:1397] T 24a91b281a7b488f8a1a6cd87e0b850c P 67abaec1b40248afb1a33088431ae1d5: Time spent bootstrapping tablet: real 0.017s user 0.012s sys 0.001s
I20260806 03:49:34.030644 1179 raft_consensus.cc:348] T 24a91b281a7b488f8a1a6cd87e0b850c P 67abaec1b40248afb1a33088431ae1d5 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "403aacadd2b546c1b7289fad1f6066eb" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 39103 } } peers { permanent_uuid: "df657246a4be41cdb398d7c7e988d8e4" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 37941 } } peers { permanent_uuid: "67abaec1b40248afb1a33088431ae1d5" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 36851 } }
I20260806 03:49:34.031005 1179 raft_consensus.cc:729] T 24a91b281a7b488f8a1a6cd87e0b850c P 67abaec1b40248afb1a33088431ae1d5 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 67abaec1b40248afb1a33088431ae1d5, State: Initialized, Role: FOLLOWER
I20260806 03:49:34.031229 1179 consensus_queue.cc:260] T 24a91b281a7b488f8a1a6cd87e0b850c P 67abaec1b40248afb1a33088431ae1d5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "403aacadd2b546c1b7289fad1f6066eb" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 39103 } } peers { permanent_uuid: "df657246a4be41cdb398d7c7e988d8e4" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 37941 } } peers { permanent_uuid: "67abaec1b40248afb1a33088431ae1d5" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 36851 } }
I20260806 03:49:34.031561 1179 ts_tablet_manager.cc:1428] T 24a91b281a7b488f8a1a6cd87e0b850c P 67abaec1b40248afb1a33088431ae1d5: Time spent starting tablet: real 0.002s user 0.006s sys 0.000s
W20260806 03:49:34.116233 1336 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:49:34.116499 1336 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:49:34.116540 1336 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:49:34.117750 1336 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:49:34.117826 1336 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:49:34.117868 1336 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.253.194
I20260806 03:49:34.119385 1336 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.253.212:40573
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.194.keytab
--principal=kudu/127.11.253.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.253.194:37941
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.253.194
--webserver_port=39759
--tserver_master_addrs=127.11.253.254:44939
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1336
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.253.194
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:49:34.119683 1336 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:49:34.119942 1336 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:49:34.129535 1346 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:49:34.129633 1344 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:49:34.129848 1336 server_base.cc:1034] running on GCE node
W20260806 03:49:34.129974 1348 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:49:34.130223 1336 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:49:34.130527 1336 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:49:34.131707 1336 hybrid_clock.cc:648] HybridClock initialized: now 1785988174131684 us; error 46 us; skew 500 ppm
W20260806 03:49:34.134049 1334 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 03:49:34.020803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:44939 (local address 127.11.253.193:60305)
0806 03:49:34.020988 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0806 03:49:34.021033 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 03:49:34.021491 (+ 458us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:49:34.133790 (+112299us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.253.254:44939: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":110,"threads_started":1}
Aug 06 03:49:34 dist-test-slave-6ntk krb5kdc[31917](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988174, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.253.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260806 03:49:34.134339 1332 heartbeater.cc:644] Failed to heartbeat to 127.11.253.254:44939 (0 consecutive failures): Network error: Failed to ping master at 127.11.253.254:44939: Client connection negotiation failed: client connection to 127.11.253.254:44939: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260806 03:49:34.134776 1334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:49:34.134594 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:44939 (local address 127.11.253.193:38449)
0806 03:49:34.134686 (+ 92us) negotiation.cc:107] Waiting for socket to connect
0806 03:49:34.134735 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.253.254:44939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260806 03:49:34.134825 1336 init.cc:377] Logged in from keytab as kudu/127.11.253.194@KRBTEST.COM (short username kudu)
I20260806 03:49:34.135139 1334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:49:34.135054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:44939 (local address 127.11.253.193:46273)
0806 03:49:34.135089 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0806 03:49:34.135119 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.253.254:44939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":20}
W20260806 03:49:34.135263 1332 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260806 03:49:34.135475 1336 webserver.cc:460] Webserver started at http://127.11.253.194:39759/ using document root <none> and password file <none>
I20260806 03:49:34.135740 1336 fs_manager.cc:359] Metadata directory not provided
I20260806 03:49:34.135820 1336 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:49:34.137187 1336 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.001s
I20260806 03:49:34.138012 1356 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:49:34.138223 1336 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260806 03:49:34.138327 1336 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/minicluster-data/ts-1/data,/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/minicluster-data/ts-1/wal
uuid: "df657246a4be41cdb398d7c7e988d8e4"
format_stamp: "Formatted at 2026-08-06 03:49:32 on dist-test-slave-6ntk"
server_key: "b713a4254580841280323bb428a9e14f"
server_key_iv: "16bdd544f6ea35651e971f1ea83e3d4e"
server_key_version: "encryptionkey@0"
I20260806 03:49:34.138692 1336 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:49:34.169847 1336 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:49:34.171942 1336 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:49:34.172116 1336 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:49:34.172370 1336 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 03:49:34.172694 1336 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 03:49:34.172750 1336 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 03:49:34.172822 1336 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 03:49:34.172854 1336 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 03:49:34.179476 1336 rpc_server.cc:307] RPC server started. Bound to: 127.11.253.194:37941
I20260806 03:49:34.179581 1515 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.253.194:37941 every 8 connection(s)
I20260806 03:49:34.179939 1336 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/minicluster-data/ts-1/data/info.pb
I20260806 03:49:34.181166 1520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:49:34.180734 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:44939 (local address 127.11.253.194:51495)
0806 03:49:34.180931 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0806 03:49:34.181017 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.253.254:44939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":72,"threads_started":1}
W20260806 03:49:34.181667 1517 heartbeater.cc:644] Failed to heartbeat to 127.11.253.254:44939 (0 consecutive failures): Network error: Failed to ping master at 127.11.253.254:44939: Client connection negotiation failed: client connection to 127.11.253.254:44939: connect: Connection refused (error 111)
I20260806 03:49:34.181999 1520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:49:34.181911 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:44939 (local address 127.11.253.194:36837)
0806 03:49:34.181945 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0806 03:49:34.181979 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.253.254:44939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":17}
I20260806 03:49:34.182250 1520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:49:34.182199 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:44939 (local address 127.11.253.194:42429)
0806 03:49:34.182219 (+ 20us) negotiation.cc:107] Waiting for socket to connect
0806 03:49:34.182237 (+ 18us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.253.254:44939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":12}
W20260806 03:49:34.182410 1517 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260806 03:49:34.185993 12279 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskToVWRe/build/release/bin/kudu as pid 1336
I20260806 03:49:34.186348 12279 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskToVWRe/build/release/bin/kudu
/tmp/dist-test-taskToVWRe/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.253.195:39103
--local_ip_for_outbound_sockets=127.11.253.195
--tserver_master_addrs=127.11.253.254:44939
--webserver_port=43749
--webserver_interface=127.11.253.195
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.195.keytab
--principal=kudu/127.11.253.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.253.212:40573
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:49:34.273452 1525 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:49:34.273663 1525 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:49:34.273689 1525 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:49:34.274909 1525 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:49:34.274955 1525 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:49:34.274981 1525 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.253.195
I20260806 03:49:34.276515 1525 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.253.212:40573
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.195.keytab
--principal=kudu/127.11.253.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.253.195:39103
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.253.195
--webserver_port=43749
--tserver_master_addrs=127.11.253.254:44939
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1525
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.253.195
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:49:34.276777 1525 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:49:34.277056 1525 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:49:34.282526 1531 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:49:34.282558 1530 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:49:34.282562 1533 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:49:34.282753 1525 server_base.cc:1034] running on GCE node
I20260806 03:49:34.282999 1525 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:49:34.283265 1525 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:49:34.284431 1525 hybrid_clock.cc:648] HybridClock initialized: now 1785988174284422 us; error 38 us; skew 500 ppm
Aug 06 03:49:34 dist-test-slave-6ntk krb5kdc[31917](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988174, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.253.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:49:34.287093 1525 init.cc:377] Logged in from keytab as kudu/127.11.253.195@KRBTEST.COM (short username kudu)
I20260806 03:49:34.287570 1525 webserver.cc:460] Webserver started at http://127.11.253.195:43749/ using document root <none> and password file <none>
I20260806 03:49:34.287817 1525 fs_manager.cc:359] Metadata directory not provided
I20260806 03:49:34.287890 1525 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:49:34.289300 1525 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 03:49:34.290045 1540 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:49:34.290241 1525 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 03:49:34.290373 1525 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/minicluster-data/ts-2/data,/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/minicluster-data/ts-2/wal
uuid: "403aacadd2b546c1b7289fad1f6066eb"
format_stamp: "Formatted at 2026-08-06 03:49:32 on dist-test-slave-6ntk"
server_key: "4d1b136f6886e0b91ec96dacee8cc8da"
server_key_iv: "66f109e8c54a9e4fbe7df92edd2efe14"
server_key_version: "encryptionkey@0"
I20260806 03:49:34.290748 1525 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:49:34.314450 1525 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:49:34.316504 1525 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:49:34.316682 1525 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:49:34.316972 1525 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 03:49:34.317308 1525 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 03:49:34.317365 1525 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 03:49:34.317401 1525 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 03:49:34.317449 1525 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 03:49:34.324072 1525 rpc_server.cc:307] RPC server started. Bound to: 127.11.253.195:39103
I20260806 03:49:34.324117 1691 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.253.195:39103 every 8 connection(s)
I20260806 03:49:34.324522 1525 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988004637995-12279-0/minicluster-data/ts-2/data/info.pb
I20260806 03:49:34.325680 1696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:49:34.325256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:44939 (local address 127.11.253.195:50345)
0806 03:49:34.325436 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0806 03:49:34.325517 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.253.254:44939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":86,"threads_started":1}
W20260806 03:49:34.326217 1692 heartbeater.cc:644] Failed to heartbeat to 127.11.253.254:44939 (0 consecutive failures): Network error: Failed to ping master at 127.11.253.254:44939: Client connection negotiation failed: client connection to 127.11.253.254:44939: connect: Connection refused (error 111)
I20260806 03:49:34.326525 1696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:49:34.326424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:44939 (local address 127.11.253.195:40779)
0806 03:49:34.326471 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0806 03:49:34.326503 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.253.254:44939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260806 03:49:34.326851 1696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:49:34.326774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:44939 (local address 127.11.253.195:41041)
0806 03:49:34.326811 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0806 03:49:34.326834 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.253.254:44939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260806 03:49:34.326975 1692 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260806 03:49:34.334381 12279 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskToVWRe/build/release/bin/kudu as pid 1525
W20260806 03:49:35.021207 1327 messenger.cc:188] Invalid argument: socket address is ambiguous: no data on pending connections for acceptor pool at 127.11.253.193:36851
I20260806 03:49:35.135980 1698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:49:35.135659 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:44939 (local address 127.11.253.193:44883)
0806 03:49:35.135862 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0806 03:49:35.135910 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.253.254:44939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":69,"threads_started":1}
I20260806 03:49:35.182976 1700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:49:35.182754 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:44939 (local address 127.11.253.194:58637)
0806 03:49:35.182878 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0806 03:49:35.182920 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.253.254:44939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":67,"threads_started":1}
I20260806 03:49:35.327551 1701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:49:35.327339 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:44939 (local address 127.11.253.195:44215)
0806 03:49:35.327469 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0806 03:49:35.327503 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.253.254:44939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":74,"threads_started":1}
I20260806 03:49:35.544335 1702 raft_consensus.cc:482] T 24a91b281a7b488f8a1a6cd87e0b850c P 67abaec1b40248afb1a33088431ae1d5 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 03:49:35.544479 1702 raft_consensus.cc:504] T 24a91b281a7b488f8a1a6cd87e0b850c P 67abaec1b40248afb1a33088431ae1d5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "403aacadd2b546c1b7289fad1f6066eb" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 39103 } } peers { permanent_uuid: "df657246a4be41cdb398d7c7e988d8e4" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 37941 } } peers { permanent_uuid: "67abaec1b40248afb1a33088431ae1d5" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 36851 } }
I20260806 03:49:35.544900 1702 leader_election.cc:288] T 24a91b281a7b488f8a1a6cd87e0b850c P 67abaec1b40248afb1a33088431ae1d5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 403aacadd2b546c1b7289fad1f6066eb (127.11.253.195:39103), df657246a4be41cdb398d7c7e988d8e4 (127.11.253.194:37941)
Aug 06 03:49:35 dist-test-slave-6ntk krb5kdc[31917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988173, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.253.193@KRBTEST.COM for kudu/127.11.253.194@KRBTEST.COM
Aug 06 03:49:35 dist-test-slave-6ntk krb5kdc[31917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988173, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.253.193@KRBTEST.COM for kudu/127.11.253.195@KRBTEST.COM
I20260806 03:49:35.559446 1708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:49:35.545383 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.253.193:36343 (local address 127.11.253.194:37941)
0806 03:49:35.545544 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0806 03:49:35.545552 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 03:49:35.545581 (+ 29us) server_negotiation.cc:407] Connection header received
0806 03:49:35.546223 (+ 642us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:49:35.546225 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:49:35.546282 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:49:35.546443 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:49:35.547504 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:49:35.548451 (+ 947us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:49:35.550042 (+ 1591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:49:35.550711 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:49:35.550844 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:49:35.554293 (+ 3449us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:49:35.554298 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:49:35.554298 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:49:35.554312 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:49:35.556700 (+ 2388us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:49:35.557474 (+ 774us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:49:35.557478 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:49:35.557479 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:49:35.557646 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:49:35.558209 (+ 563us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:49:35.558211 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:49:35.558211 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:49:35.558521 (+ 310us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:49:35.558615 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0806 03:49:35.559149 (+ 534us) server_negotiation.cc:299] Negotiation successful
0806 03:49:35.559341 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":79,"threads_started":1}
I20260806 03:49:35.559446 1704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:49:35.545356 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.253.193:58441 (local address 127.11.253.195:39103)
0806 03:49:35.545509 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0806 03:49:35.545524 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 03:49:35.545565 (+ 41us) server_negotiation.cc:407] Connection header received
0806 03:49:35.546214 (+ 649us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:49:35.546218 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:49:35.546282 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:49:35.546443 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:49:35.547490 (+ 1047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:49:35.548543 (+ 1053us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:49:35.550042 (+ 1499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:49:35.550708 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:49:35.550844 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:49:35.554582 (+ 3738us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:49:35.554587 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:49:35.554587 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:49:35.554602 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:49:35.556713 (+ 2111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:49:35.557474 (+ 761us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:49:35.557478 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:49:35.557479 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:49:35.557616 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:49:35.558116 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:49:35.558118 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:49:35.558118 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:49:35.558521 (+ 403us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:49:35.558615 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0806 03:49:35.559149 (+ 534us) server_negotiation.cc:299] Negotiation successful
0806 03:49:35.559315 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108,"thread_start_us":53,"threads_started":1}
I20260806 03:49:35.559470 1703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:49:35.545299 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.194:37941 (local address 127.11.253.193:36343)
0806 03:49:35.545411 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0806 03:49:35.545424 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 03:49:35.545482 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:49:35.546520 (+ 1038us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:49:35.546523 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:49:35.547061 (+ 538us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:49:35.547348 (+ 287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:49:35.547356 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:49:35.548569 (+ 1213us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:49:35.548570 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:49:35.549942 (+ 1372us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:49:35.549945 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:49:35.550848 (+ 903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:49:35.550849 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:49:35.551020 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:49:35.551805 (+ 785us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:49:35.551811 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:49:35.554121 (+ 2310us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:49:35.556827 (+ 2706us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:49:35.556840 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:49:35.556843 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:49:35.557344 (+ 501us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:49:35.557744 (+ 400us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:49:35.557745 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:49:35.557746 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:49:35.558120 (+ 374us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:49:35.558631 (+ 511us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:49:35.558649 (+ 18us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:49:35.558850 (+ 201us) client_negotiation.cc:769] Sending connection context
0806 03:49:35.559010 (+ 160us) client_negotiation.cc:240] Negotiation successful
0806 03:49:35.559328 (+ 318us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":98,"mutex_wait_us":105,"thread_start_us":65,"threads_started":1}
I20260806 03:49:35.560151 1698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:49:35.545143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.195:39103 (local address 127.11.253.193:58441)
0806 03:49:35.545206 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0806 03:49:35.545221 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 03:49:35.545527 (+ 306us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:49:35.546520 (+ 993us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:49:35.546523 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:49:35.547061 (+ 538us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:49:35.547348 (+ 287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:49:35.547356 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:49:35.548630 (+ 1274us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:49:35.548630 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:49:35.549942 (+ 1312us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:49:35.549945 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:49:35.550847 (+ 902us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:49:35.550849 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:49:35.551020 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:49:35.551795 (+ 775us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:49:35.551811 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:49:35.554448 (+ 2637us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:49:35.556827 (+ 2379us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:49:35.556840 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:49:35.556843 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:49:35.557356 (+ 513us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:49:35.557754 (+ 398us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:49:35.557759 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:49:35.557760 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:49:35.558020 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:49:35.558631 (+ 611us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:49:35.558642 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:49:35.558851 (+ 209us) client_negotiation.cc:769] Sending connection context
0806 03:49:35.559010 (+ 159us) client_negotiation.cc:240] Negotiation successful
0806 03:49:35.559403 (+ 393us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36,"mutex_wait_us":294}
W20260806 03:49:35.560617 1172 leader_election.cc:341] T 24a91b281a7b488f8a1a6cd87e0b850c P 67abaec1b40248afb1a33088431ae1d5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer df657246a4be41cdb398d7c7e988d8e4 (127.11.253.194:37941): Not found: Tablet not found: 24a91b281a7b488f8a1a6cd87e0b850c
W20260806 03:49:35.561066 1172 leader_election.cc:341] T 24a91b281a7b488f8a1a6cd87e0b850c P 67abaec1b40248afb1a33088431ae1d5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 403aacadd2b546c1b7289fad1f6066eb (127.11.253.195:39103): Not found: Tablet not found: 24a91b281a7b488f8a1a6cd87e0b850c
I20260806 03:49:35.561131 1172 leader_election.cc:302] T 24a91b281a7b488f8a1a6cd87e0b850c P 67abaec1b40248afb1a33088431ae1d5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 67abaec1b40248afb1a33088431ae1d5; no voters: 403aacadd2b546c1b7289fad1f6066eb, df657246a4be41cdb398d7c7e988d8e4
I20260806 03:49:35.561276 1702 raft_consensus.cc:2726] T 24a91b281a7b488f8a1a6cd87e0b850c P 67abaec1b40248afb1a33088431ae1d5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 03:49:36.136839 1709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:49:36.136579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:44939 (local address 127.11.253.193:37873)
0806 03:49:36.136744 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0806 03:49:36.136787 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.253.254:44939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":67,"threads_started":1}
I20260806 03:49:36.183688 1710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:49:36.183467 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:44939 (local address 127.11.253.194:48823)
0806 03:49:36.183606 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0806 03:49:36.183645 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.253.254:44939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":64,"threads_started":1}
I20260806 03:49:36.328291 1711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:49:36.328059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:44939 (local address 127.11.253.195:40903)
0806 03:49:36.328207 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0806 03:49:36.328247 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.253.254:44939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":73,"threads_started":1}
I20260806 03:49:37.137828 1712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:49:37.137513 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:44939 (local address 127.11.253.193:46237)
0806 03:49:37.137720 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0806 03:49:37.137771 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.253.254:44939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":91,"threads_started":1}
I20260806 03:49:37.184471 1713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:49:37.184182 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:44939 (local address 127.11.253.194:45815)
0806 03:49:37.184387 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0806 03:49:37.184426 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.253.254:44939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":118,"threads_started":1}
I20260806 03:49:37.329034 1714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:49:37.328818 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:44939 (local address 127.11.253.195:53089)
0806 03:49:37.328957 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0806 03:49:37.328994 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.253.254:44939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":74,"threads_started":1}
I20260806 03:49:37.614564 1717 raft_consensus.cc:482] T 24a91b281a7b488f8a1a6cd87e0b850c P 67abaec1b40248afb1a33088431ae1d5 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 03:49:37.614672 1717 raft_consensus.cc:504] T 24a91b281a7b488f8a1a6cd87e0b850c P 67abaec1b40248afb1a33088431ae1d5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "403aacadd2b546c1b7289fad1f6066eb" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 39103 } } peers { permanent_uuid: "df657246a4be41cdb398d7c7e988d8e4" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 37941 } } peers { permanent_uuid: "67abaec1b40248afb1a33088431ae1d5" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 36851 } }
I20260806 03:49:37.614907 1717 leader_election.cc:288] T 24a91b281a7b488f8a1a6cd87e0b850c P 67abaec1b40248afb1a33088431ae1d5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 403aacadd2b546c1b7289fad1f6066eb (127.11.253.195:39103), df657246a4be41cdb398d7c7e988d8e4 (127.11.253.194:37941)
W20260806 03:49:37.615401 1172 leader_election.cc:341] T 24a91b281a7b488f8a1a6cd87e0b850c P 67abaec1b40248afb1a33088431ae1d5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 403aacadd2b546c1b7289fad1f6066eb (127.11.253.195:39103): Not found: Tablet not found: 24a91b281a7b488f8a1a6cd87e0b850c
W20260806 03:49:37.615513 1172 leader_election.cc:341] T 24a91b281a7b488f8a1a6cd87e0b850c P 67abaec1b40248afb1a33088431ae1d5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer df657246a4be41cdb398d7c7e988d8e4 (127.11.253.194:37941): Not found: Tablet not found: 24a91b281a7b488f8a1a6cd87e0b850c
I20260806 03:49:37.615550 1172 leader_election.cc:302] T 24a91b281a7b488f8a1a6cd87e0b850c P 67abaec1b40248afb1a33088431ae1d5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 67abaec1b40248afb1a33088431ae1d5; no voters: 403aacadd2b546c1b7289fad1f6066eb, df657246a4be41cdb398d7c7e988d8e4
I20260806 03:49:37.615645 1717 raft_consensus.cc:2726] T 24a91b281a7b488f8a1a6cd87e0b850c P 67abaec1b40248afb1a33088431ae1d5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 03:49:38.138842 1718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:49:38.138593 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:44939 (local address 127.11.253.193:52801)
0806 03:49:38.138745 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0806 03:49:38.138792 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.253.254:44939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":95,"threads_started":1}
I20260806 03:49:38.185405 1719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:49:38.184994 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:44939 (local address 127.11.253.194:54371)
0806 03:49:38.185220 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0806 03:49:38.185329 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.253.254:44939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":73,"threads_started":1}
I20260806 03:49:38.329800 1720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:49:38.329563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:44939 (local address 127.11.253.195:36183)
0806 03:49:38.329712 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0806 03:49:38.329755 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.253.254:44939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":69,"threads_started":1}
I20260806 03:49:39.139742 1722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:49:39.139486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:44939 (local address 127.11.253.193:38869)
0806 03:49:39.139653 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0806 03:49:39.139692 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.253.254:44939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":81,"threads_started":1}
I20260806 03:49:39.186331 1724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:49:39.186053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:44939 (local address 127.11.253.194:36737)
0806 03:49:39.186230 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0806 03:49:39.186271 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.253.254:44939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":96,"threads_started":1}
I20260806 03:49:39.330559 1725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:49:39.330313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:44939 (local address 127.11.253.195:37459)
0806 03:49:39.330479 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0806 03:49:39.330515 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.253.254:44939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":103,"threads_started":1}
I20260806 03:49:39.435165 1726 raft_consensus.cc:482] T 24a91b281a7b488f8a1a6cd87e0b850c P 67abaec1b40248afb1a33088431ae1d5 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 03:49:39.435345 1726 raft_consensus.cc:504] T 24a91b281a7b488f8a1a6cd87e0b850c P 67abaec1b40248afb1a33088431ae1d5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "403aacadd2b546c1b7289fad1f6066eb" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 39103 } } peers { permanent_uuid: "df657246a4be41cdb398d7c7e988d8e4" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 37941 } } peers { permanent_uuid: "67abaec1b40248afb1a33088431ae1d5" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 36851 } }
I20260806 03:49:39.435549 1726 leader_election.cc:288] T 24a91b281a7b488f8a1a6cd87e0b850c P 67abaec1b40248afb1a33088431ae1d5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 403aacadd2b546c1b7289fad1f6066eb (127.11.253.195:39103), df657246a4be41cdb398d7c7e988d8e4 (127.11.253.194:37941)
W20260806 03:49:39.435988 1172 leader_election.cc:341] T 24a91b281a7b488f8a1a6cd87e0b850c P 67abaec1b40248afb1a33088431ae1d5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer df657246a4be41cdb398d7c7e988d8e4 (127.11.253.194:37941): Not found: Tablet not found: 24a91b281a7b488f8a1a6cd87e0b850c
W20260806 03:49:39.436074 1172 leader_election.cc:341] T 24a91b281a7b488f8a1a6cd87e0b850c P 67abaec1b40248afb1a33088431ae1d5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 403aacadd2b546c1b7289fad1f6066eb (127.11.253.195:39103): Not found: Tablet not found: 24a91b281a7b488f8a1a6cd87e0b850c
I20260806 03:49:39.436090 1172 leader_election.cc:302] T 24a91b281a7b488f8a1a6cd87e0b850c P 67abaec1b40248afb1a33088431ae1d5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 67abaec1b40248afb1a33088431ae1d5; no voters: 403aacadd2b546c1b7289fad1f6066eb, df657246a4be41cdb398d7c7e988d8e4
I20260806 03:49:39.436205 1726 raft_consensus.cc:2726] T 24a91b281a7b488f8a1a6cd87e0b850c P 67abaec1b40248afb1a33088431ae1d5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 03:49:40.140581 1727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:49:40.140345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:44939 (local address 127.11.253.193:49633)
0806 03:49:40.140486 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0806 03:49:40.140528 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.253.254:44939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":76,"threads_started":1}
I20260806 03:49:40.187279 1728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:49:40.186960 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:44939 (local address 127.11.253.194:58881)
0806 03:49:40.187158 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0806 03:49:40.187211 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.253.254:44939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":116,"threads_started":1}
I20260806 03:49:40.331316 1729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:49:40.331038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:44939 (local address 127.11.253.195:49895)
0806 03:49:40.331196 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0806 03:49:40.331265 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.253.254:44939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":64,"threads_started":1}
I20260806 03:49:41.141538 1730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:49:41.141273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:44939 (local address 127.11.253.193:42467)
0806 03:49:41.141441 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0806 03:49:41.141487 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.253.254:44939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":72,"threads_started":1}
I20260806 03:49:41.188077 1731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:49:41.187850 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:44939 (local address 127.11.253.194:36363)
0806 03:49:41.187986 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0806 03:49:41.188026 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.253.254:44939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":70,"threads_started":1}
I20260806 03:49:41.332208 1732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:49:41.331956 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:44939 (local address 127.11.253.195:34471)
0806 03:49:41.332115 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0806 03:49:41.332157 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.253.254:44939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":67,"threads_started":1}
I20260806 03:49:41.918619 1733 raft_consensus.cc:482] T 24a91b281a7b488f8a1a6cd87e0b850c P 67abaec1b40248afb1a33088431ae1d5 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 03:49:41.918725 1733 raft_consensus.cc:504] T 24a91b281a7b488f8a1a6cd87e0b850c P 67abaec1b40248afb1a33088431ae1d5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "403aacadd2b546c1b7289fad1f6066eb" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 39103 } } peers { permanent_uuid: "df657246a4be41cdb398d7c7e988d8e4" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 37941 } } peers { permanent_uuid: "67abaec1b40248afb1a33088431ae1d5" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 36851 } }
I20260806 03:49:41.918861 1733 leader_election.cc:288] T 24a91b281a7b488f8a1a6cd87e0b850c P 67abaec1b40248afb1a33088431ae1d5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 403aacadd2b546c1b7289fad1f6066eb (127.11.253.195:39103), df657246a4be41cdb398d7c7e988d8e4 (127.11.253.194:37941)
W20260806 03:49:41.919363 1172 leader_election.cc:341] T 24a91b281a7b488f8a1a6cd87e0b850c P 67abaec1b40248afb1a33088431ae1d5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 403aacadd2b546c1b7289fad1f6066eb (127.11.253.195:39103): Not found: Tablet not found: 24a91b281a7b488f8a1a6cd87e0b850c
W20260806 03:49:41.919462 1172 leader_election.cc:341] T 24a91b281a7b488f8a1a6cd87e0b850c P 67abaec1b40248afb1a33088431ae1d5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer df657246a4be41cdb398d7c7e988d8e4 (127.11.253.194:37941): Not found: Tablet not found: 24a91b281a7b488f8a1a6cd87e0b850c
I20260806 03:49:41.919513 1172 leader_election.cc:302] T 24a91b281a7b488f8a1a6cd87e0b850c P 67abaec1b40248afb1a33088431ae1d5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 67abaec1b40248afb1a33088431ae1d5; no voters: 403aacadd2b546c1b7289fad1f6066eb, df657246a4be41cdb398d7c7e988d8e4
I20260806 03:49:41.919601 1733 raft_consensus.cc:2726] T 24a91b281a7b488f8a1a6cd87e0b850c P 67abaec1b40248afb1a33088431ae1d5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 03:49:42.142413 1734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:49:42.142133 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:44939 (local address 127.11.253.193:35157)
0806 03:49:42.142300 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0806 03:49:42.142343 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.253.254:44939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":84,"threads_started":1}
I20260806 03:49:42.188853 1735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:49:42.188569 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:44939 (local address 127.11.253.194:60225)
0806 03:49:42.188720 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0806 03:49:42.188783 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.253.254:44939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":58,"threads_started":1}
I20260806 03:49:42.332995 1737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:49:42.332774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:44939 (local address 127.11.253.195:42285)
0806 03:49:42.332909 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0806 03:49:42.332947 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.253.254:44939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":73,"threads_started":1}
I20260806 03:49:43.143265 1738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:49:43.142957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:44939 (local address 127.11.253.193:47199)
0806 03:49:43.143143 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0806 03:49:43.143209 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.253.254:44939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":77,"threads_started":1}
I20260806 03:49:43.189605 1741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:49:43.189377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:44939 (local address 127.11.253.194:56173)
0806 03:49:43.189517 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0806 03:49:43.189557 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.253.254:44939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":61,"threads_started":1}
I20260806 03:49:43.333922 1743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:49:43.333608 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:44939 (local address 127.11.253.195:46807)
0806 03:49:43.333779 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0806 03:49:43.333827 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.253.254:44939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":93,"threads_started":1}
I20260806 03:49:44.144052 1744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:49:44.143798 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:44939 (local address 127.11.253.193:32777)
0806 03:49:44.143957 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0806 03:49:44.144000 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.253.254:44939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":87,"threads_started":1}
I20260806 03:49:44.190485 1745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:49:44.190196 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:44939 (local address 127.11.253.194:33917)
0806 03:49:44.190369 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0806 03:49:44.190421 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.253.254:44939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":99,"threads_started":1}
I20260806 03:49:44.327653 1747 raft_consensus.cc:482] T 24a91b281a7b488f8a1a6cd87e0b850c P 67abaec1b40248afb1a33088431ae1d5 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 03:49:44.327785 1747 raft_consensus.cc:504] T 24a91b281a7b488f8a1a6cd87e0b850c P 67abaec1b40248afb1a33088431ae1d5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "403aacadd2b546c1b7289fad1f6066eb" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 39103 } } peers { permanent_uuid: "df657246a4be41cdb398d7c7e988d8e4" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 37941 } } peers { permanent_uuid: "67abaec1b40248afb1a33088431ae1d5" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 36851 } }
I20260806 03:49:44.327991 1747 leader_election.cc:288] T 24a91b281a7b488f8a1a6cd87e0b850c P 67abaec1b40248afb1a33088431ae1d5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 403aacadd2b546c1b7289fad1f6066eb (127.11.253.195:39103), df657246a4be41cdb398d7c7e988d8e4 (127.11.253.194:37941)
W20260806 03:49:44.328459 1172 leader_election.cc:341] T 24a91b281a7b488f8a1a6cd87e0b850c P 67abaec1b40248afb1a33088431ae1d5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer df657246a4be41cdb398d7c7e988d8e4 (127.11.253.194:37941): Not found: Tablet not found: 24a91b281a7b488f8a1a6cd87e0b850c
W20260806 03:49:44.328539 1172 leader_election.cc:341] T 24a91b281a7b488f8a1a6cd87e0b850c P 67abaec1b40248afb1a33088431ae1d5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 403aacadd2b546c1b7289fad1f6066eb (127.11.253.195:39103): Not found: Tablet not found: 24a91b281a7b488f8a1a6cd87e0b850c
I20260806 03:49:44.328557 1172 leader_election.cc:302] T 24a91b281a7b488f8a1a6cd87e0b850c P 67abaec1b40248afb1a33088431ae1d5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 67abaec1b40248afb1a33088431ae1d5; no voters: 403aacadd2b546c1b7289fad1f6066eb, df657246a4be41cdb398d7c7e988d8e4
I20260806 03:49:44.328672 1747 raft_consensus.cc:2726] T 24a91b281a7b488f8a1a6cd87e0b850c P 67abaec1b40248afb1a33088431ae1d5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 03:49:44.334690 1748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:49:44.334441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:44939 (local address 127.11.253.195:50177)
0806 03:49:44.334600 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0806 03:49:44.334643 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.253.254:44939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":65,"threads_started":1}
I20260806 03:49:45.144800 1749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:49:45.144554 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:44939 (local address 127.11.253.193:32879)
0806 03:49:45.144706 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0806 03:49:45.144748 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.253.254:44939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":86,"threads_started":1}
I20260806 03:49:45.191149 1750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:49:45.190930 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:44939 (local address 127.11.253.194:46815)
0806 03:49:45.191057 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0806 03:49:45.191099 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.253.254:44939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":64,"threads_started":1}
I20260806 03:49:45.335427 1751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:49:45.335199 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:44939 (local address 127.11.253.195:45839)
0806 03:49:45.335336 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0806 03:49:45.335377 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.253.254:44939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":64,"threads_started":1}
I20260806 03:49:46.145684 1752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:49:46.145387 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:44939 (local address 127.11.253.193:56069)
0806 03:49:46.145568 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0806 03:49:46.145620 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.253.254:44939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":76,"threads_started":1}
I20260806 03:49:46.192011 1753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:49:46.191712 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:44939 (local address 127.11.253.194:60769)
0806 03:49:46.191887 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0806 03:49:46.191944 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.253.254:44939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":94,"threads_started":1}
I20260806 03:49:46.336230 1754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:49:46.335984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:44939 (local address 127.11.253.195:48201)
0806 03:49:46.336134 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0806 03:49:46.336181 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.253.254:44939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":71,"threads_started":1}
I20260806 03:49:47.146590 1758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:49:47.146312 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:44939 (local address 127.11.253.193:56293)
0806 03:49:47.146487 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0806 03:49:47.146537 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.253.254:44939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":76,"threads_started":1}
I20260806 03:49:47.192870 1760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:49:47.192603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:44939 (local address 127.11.253.194:50733)
0806 03:49:47.192783 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0806 03:49:47.192821 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.253.254:44939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":104,"threads_started":1}
I20260806 03:49:47.337105 1762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:49:47.336830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:44939 (local address 127.11.253.195:51203)
0806 03:49:47.337008 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0806 03:49:47.337054 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.253.254:44939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":85,"threads_started":1}
I20260806 03:49:48.147536 1763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:49:48.147245 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:44939 (local address 127.11.253.193:57879)
0806 03:49:48.147436 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0806 03:49:48.147483 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.253.254:44939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":75,"threads_started":1}
I20260806 03:49:48.193749 1764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:49:48.193434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:44939 (local address 127.11.253.194:53729)
0806 03:49:48.193594 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0806 03:49:48.193678 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.253.254:44939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":92,"threads_started":1}
I20260806 03:49:48.338238 1765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:49:48.337995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:44939 (local address 127.11.253.195:53225)
0806 03:49:48.338148 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0806 03:49:48.338184 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.253.254:44939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":89,"threads_started":1}
I20260806 03:49:49.148424 1766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:49:49.148180 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:44939 (local address 127.11.253.193:37027)
0806 03:49:49.148337 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0806 03:49:49.148377 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.253.254:44939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":93,"threads_started":1}
I20260806 03:49:49.194554 1768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:49:49.194292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:44939 (local address 127.11.253.194:55789)
0806 03:49:49.194465 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0806 03:49:49.194507 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.253.254:44939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":108,"threads_started":1}
I20260806 03:49:49.335657 12279 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskToVWRe/build/release/bin/kudu with pid 1099
I20260806 03:49:49.340157 1769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:49:49.339323 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:44939 (local address 127.11.253.195:52339)
0806 03:49:49.340060 (+ 737us) negotiation.cc:107] Waiting for socket to connect
0806 03:49:49.340096 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.253.254:44939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":718,"thread_start_us":668,"threads_started":1}
I20260806 03:49:49.348785 12279 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskToVWRe/build/release/bin/kudu with pid 1336
I20260806 03:49:49.356943 12279 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskToVWRe/build/release/bin/kudu with pid 1525
2026-08-06T03:49:49Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (18846 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 03:49:49 dist-test-slave-6ntk krb5kdc[1775](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 03:49:49 dist-test-slave-6ntk krb5kdc[1775](info): set up 2 sockets
Aug 06 03:49:49 dist-test-slave-6ntk krb5kdc[1775](info): commencing operation
krb5kdc: starting...
W20260806 03:49:50.597134 12279 mini_kdc.cc:121] Time spent starting KDC: real 1.216s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/krb5kdc/test-user.keytab.
Aug 06 03:49:50 dist-test-slave-6ntk krb5kdc[1775](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988190, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T03:49:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T03:49:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.253.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.253.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.253.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.254.keytab.
Entry for principal kudu/127.11.253.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.254.keytab.
I20260806 03:49:50.729669 12279 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskToVWRe/build/release/bin/kudu
/tmp/dist-test-taskToVWRe/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.253.254:42435
--webserver_interface=127.11.253.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.254.keytab
--principal=kudu/127.11.253.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.253.212:40037
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.253.254:42435
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:49:50.812884 1795 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:49:50.813135 1795 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:49:50.813174 1795 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:49:50.814330 1795 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260806 03:49:50.814409 1795 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260806 03:49:50.814436 1795 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260806 03:49:50.814464 1795 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 03:49:50.814499 1795 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:49:50.814528 1795 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:49:50.814555 1795 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 03:49:50.814582 1795 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 03:49:50.816056 1795 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.253.212:40037
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.11.253.254:42435
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.254.keytab
--principal=kudu/127.11.253.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.253.254:42435
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.253.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1795
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:49:50.816345 1795 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:49:50.816591 1795 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 03:49:50.821741 1795 server_base.cc:1034] running on GCE node
W20260806 03:49:50.821704 1802 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:49:50.821697 1807 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:49:50.821698 1803 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:49:50.822310 1795 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:49:50.822618 1795 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:49:50.823764 1795 hybrid_clock.cc:648] HybridClock initialized: now 1785988190823752 us; error 30 us; skew 500 ppm
Aug 06 03:49:50 dist-test-slave-6ntk krb5kdc[1775](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988190, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.253.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:49:50.826402 1795 init.cc:377] Logged in from keytab as kudu/127.11.253.254@KRBTEST.COM (short username kudu)
I20260806 03:49:50.826900 1795 webserver.cc:460] Webserver started at http://127.11.253.254:39741/ using document root <none> and password file <none>
I20260806 03:49:50.827162 1795 fs_manager.cc:359] Metadata directory not provided
I20260806 03:49:50.827207 1795 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:49:50.827385 1795 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:49:50.828529 1795 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/minicluster-data/master-0/data/instance:
uuid: "eba5da44f0f04fc6bdc7488532ffbd26"
format_stamp: "Formatted at 2026-08-06 03:49:50 on dist-test-slave-6ntk"
server_key: "1ca51766568e6569ae46743e4cc146f9"
server_key_iv: "28d770e821969572ffc0c2ff0cfbdb81"
server_key_version: "encryptionkey@0"
I20260806 03:49:50.828900 1795 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/minicluster-data/master-0/wal/instance:
uuid: "eba5da44f0f04fc6bdc7488532ffbd26"
format_stamp: "Formatted at 2026-08-06 03:49:50 on dist-test-slave-6ntk"
server_key: "1ca51766568e6569ae46743e4cc146f9"
server_key_iv: "28d770e821969572ffc0c2ff0cfbdb81"
server_key_version: "encryptionkey@0"
I20260806 03:49:50.830329 1795 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 03:49:50.831081 1816 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:49:50.831277 1795 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 03:49:50.831377 1795 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/minicluster-data/master-0/data,/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/minicluster-data/master-0/wal
uuid: "eba5da44f0f04fc6bdc7488532ffbd26"
format_stamp: "Formatted at 2026-08-06 03:49:50 on dist-test-slave-6ntk"
server_key: "1ca51766568e6569ae46743e4cc146f9"
server_key_iv: "28d770e821969572ffc0c2ff0cfbdb81"
server_key_version: "encryptionkey@0"
I20260806 03:49:50.831453 1795 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:49:50.841135 1795 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:49:50.843312 1795 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:49:50.843466 1795 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:49:50.847704 1795 rpc_server.cc:307] RPC server started. Bound to: 127.11.253.254:42435
I20260806 03:49:50.847769 1884 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.253.254:42435 every 8 connection(s)
I20260806 03:49:50.848104 1795 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/minicluster-data/master-0/data/info.pb
I20260806 03:49:50.849004 1886 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:49:50.851372 1886 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eba5da44f0f04fc6bdc7488532ffbd26: Bootstrap starting.
I20260806 03:49:50.851998 1886 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P eba5da44f0f04fc6bdc7488532ffbd26: Neither blocks nor log segments found. Creating new log.
I20260806 03:49:50.852301 1886 log.cc:824] T 00000000000000000000000000000000 P eba5da44f0f04fc6bdc7488532ffbd26: Log is configured to *not* fsync() on all Append() calls
I20260806 03:49:50.852995 1886 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eba5da44f0f04fc6bdc7488532ffbd26: No bootstrap required, opened a new log
I20260806 03:49:50.854672 1886 raft_consensus.cc:348] T 00000000000000000000000000000000 P eba5da44f0f04fc6bdc7488532ffbd26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eba5da44f0f04fc6bdc7488532ffbd26" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 42435 } }
I20260806 03:49:50.854811 1886 raft_consensus.cc:374] T 00000000000000000000000000000000 P eba5da44f0f04fc6bdc7488532ffbd26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:49:50.854835 1886 raft_consensus.cc:729] T 00000000000000000000000000000000 P eba5da44f0f04fc6bdc7488532ffbd26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eba5da44f0f04fc6bdc7488532ffbd26, State: Initialized, Role: FOLLOWER
I20260806 03:49:50.855028 1886 consensus_queue.cc:260] T 00000000000000000000000000000000 P eba5da44f0f04fc6bdc7488532ffbd26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eba5da44f0f04fc6bdc7488532ffbd26" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 42435 } }
I20260806 03:49:50.855115 1886 raft_consensus.cc:388] T 00000000000000000000000000000000 P eba5da44f0f04fc6bdc7488532ffbd26 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 03:49:50.855162 1886 raft_consensus.cc:482] T 00000000000000000000000000000000 P eba5da44f0f04fc6bdc7488532ffbd26 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 03:49:50.855171 12279 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskToVWRe/build/release/bin/kudu as pid 1795
I20260806 03:49:50.855237 1886 raft_consensus.cc:3037] T 00000000000000000000000000000000 P eba5da44f0f04fc6bdc7488532ffbd26 [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:49:50.855273 12279 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/minicluster-data/master-0/wal/instance
I20260806 03:49:50.855463 12279 external_mini_cluster.cc:1428] Setting key 368f3d4c7ca44f43846c5e1466eb6cd3
I20260806 03:49:50.856014 1886 raft_consensus.cc:504] T 00000000000000000000000000000000 P eba5da44f0f04fc6bdc7488532ffbd26 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eba5da44f0f04fc6bdc7488532ffbd26" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 42435 } }
I20260806 03:49:50.856151 1886 leader_election.cc:302] T 00000000000000000000000000000000 P eba5da44f0f04fc6bdc7488532ffbd26 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: eba5da44f0f04fc6bdc7488532ffbd26; no voters:
I20260806 03:49:50.856324 1886 leader_election.cc:288] T 00000000000000000000000000000000 P eba5da44f0f04fc6bdc7488532ffbd26 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 03:49:50.856541 1891 raft_consensus.cc:2781] T 00000000000000000000000000000000 P eba5da44f0f04fc6bdc7488532ffbd26 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 03:49:50.856591 1886 sys_catalog.cc:564] T 00000000000000000000000000000000 P eba5da44f0f04fc6bdc7488532ffbd26 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 03:49:50.856693 1891 raft_consensus.cc:686] T 00000000000000000000000000000000 P eba5da44f0f04fc6bdc7488532ffbd26 [term 1 LEADER]: Becoming Leader. State: Replica: eba5da44f0f04fc6bdc7488532ffbd26, State: Running, Role: LEADER
I20260806 03:49:50.856823 1891 consensus_queue.cc:237] T 00000000000000000000000000000000 P eba5da44f0f04fc6bdc7488532ffbd26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eba5da44f0f04fc6bdc7488532ffbd26" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 42435 } }
I20260806 03:49:50.857216 1892 sys_catalog.cc:455] T 00000000000000000000000000000000 P eba5da44f0f04fc6bdc7488532ffbd26 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "eba5da44f0f04fc6bdc7488532ffbd26" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eba5da44f0f04fc6bdc7488532ffbd26" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 42435 } } }
I20260806 03:49:50.857297 1892 sys_catalog.cc:458] T 00000000000000000000000000000000 P eba5da44f0f04fc6bdc7488532ffbd26 [sys.catalog]: This master's current role is: LEADER
I20260806 03:49:50.857476 1893 sys_catalog.cc:455] T 00000000000000000000000000000000 P eba5da44f0f04fc6bdc7488532ffbd26 [sys.catalog]: SysCatalogTable state changed. Reason: New leader eba5da44f0f04fc6bdc7488532ffbd26. Latest consensus state: current_term: 1 leader_uuid: "eba5da44f0f04fc6bdc7488532ffbd26" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eba5da44f0f04fc6bdc7488532ffbd26" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 42435 } } }
I20260806 03:49:50.857529 1904 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 03:49:50.857547 1893 sys_catalog.cc:458] T 00000000000000000000000000000000 P eba5da44f0f04fc6bdc7488532ffbd26 [sys.catalog]: This master's current role is: LEADER
I20260806 03:49:50.858076 1904 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 03:49:50.859728 1904 catalog_manager.cc:1347] Generated new cluster ID: 93da3e1b34ed419e8b9740addf6b9ad8
I20260806 03:49:50.859789 1904 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 03:49:50 dist-test-slave-6ntk krb5kdc[1775](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988190, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.253.254@KRBTEST.COM
I20260806 03:49:50.866489 1890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:49:50.855981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39890 (local address 127.11.253.254:42435)
0806 03:49:50.856236 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0806 03:49:50.856247 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 03:49:50.856287 (+ 40us) server_negotiation.cc:407] Connection header received
0806 03:49:50.856423 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:49:50.856430 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:49:50.856491 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:49:50.857548 (+ 1057us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:49:50.857659 (+ 111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:49:50.859308 (+ 1649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:49:50.860236 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:49:50.860656 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:49:50.860775 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:49:50.863213 (+ 2438us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:49:50.863217 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:49:50.863218 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:49:50.863230 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:49:50.864929 (+ 1699us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:49:50.865296 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:49:50.865299 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:49:50.865300 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:49:50.865413 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:49:50.865627 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:49:50.865628 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:49:50.865629 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:49:50.865863 (+ 234us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:49:50.866089 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0806 03:49:50.866158 (+ 69us) server_negotiation.cc:299] Negotiation successful
0806 03:49:50.866295 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":205,"thread_start_us":95,"threads_started":1}
I20260806 03:49:50.885843 1904 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 03:49:50.886471 1904 catalog_manager.cc:1504] Loading token signing keys...
I20260806 03:49:50.889717 1904 catalog_manager.cc:5868] T 00000000000000000000000000000000 P eba5da44f0f04fc6bdc7488532ffbd26: Generated new TSK 0
I20260806 03:49:50.889976 1904 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.253.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.253.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.253.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.193.keytab.
Entry for principal kudu/127.11.253.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.193.keytab.
I20260806 03:49:50.921753 12279 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskToVWRe/build/release/bin/kudu
/tmp/dist-test-taskToVWRe/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.253.193:0
--local_ip_for_outbound_sockets=127.11.253.193
--webserver_interface=127.11.253.193
--webserver_port=0
--tserver_master_addrs=127.11.253.254:42435
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.193.keytab
--principal=kudu/127.11.253.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.253.212:40037
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:49:51.007794 1924 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:49:51.008010 1924 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:49:51.008034 1924 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:49:51.009130 1924 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:49:51.009166 1924 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:49:51.009192 1924 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.253.193
I20260806 03:49:51.010667 1924 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.253.212:40037
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.193.keytab
--principal=kudu/127.11.253.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.253.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.253.193
--webserver_port=0
--tserver_master_addrs=127.11.253.254:42435
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1924
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.253.193
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:49:51.010862 1924 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:49:51.011040 1924 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:49:51.016281 1933 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:49:51.016311 1930 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:49:51.016281 1929 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:49:51.016343 1924 server_base.cc:1034] running on GCE node
I20260806 03:49:51.016709 1924 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:49:51.017019 1924 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:49:51.018164 1924 hybrid_clock.cc:648] HybridClock initialized: now 1785988191018141 us; error 42 us; skew 500 ppm
Aug 06 03:49:51 dist-test-slave-6ntk krb5kdc[1775](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988191, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.253.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:49:51.020793 1924 init.cc:377] Logged in from keytab as kudu/127.11.253.193@KRBTEST.COM (short username kudu)
I20260806 03:49:51.021193 1924 webserver.cc:460] Webserver started at http://127.11.253.193:44441/ using document root <none> and password file <none>
I20260806 03:49:51.021432 1924 fs_manager.cc:359] Metadata directory not provided
I20260806 03:49:51.021546 1924 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:49:51.021708 1924 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:49:51.022837 1924 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/minicluster-data/ts-0/data/instance:
uuid: "3f2f80aff6824a9a9a5601ac2f1732c9"
format_stamp: "Formatted at 2026-08-06 03:49:51 on dist-test-slave-6ntk"
server_key: "057eb8121ad3c406a7c4008f8251437d"
server_key_iv: "0067d601f4f89d4feec5d3d1d9fa3f33"
server_key_version: "encryptionkey@0"
I20260806 03:49:51.023226 1924 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/minicluster-data/ts-0/wal/instance:
uuid: "3f2f80aff6824a9a9a5601ac2f1732c9"
format_stamp: "Formatted at 2026-08-06 03:49:51 on dist-test-slave-6ntk"
server_key: "057eb8121ad3c406a7c4008f8251437d"
server_key_iv: "0067d601f4f89d4feec5d3d1d9fa3f33"
server_key_version: "encryptionkey@0"
I20260806 03:49:51.024616 1924 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 03:49:51.025375 1941 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:49:51.025557 1924 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 03:49:51.025668 1924 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/minicluster-data/ts-0/data,/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/minicluster-data/ts-0/wal
uuid: "3f2f80aff6824a9a9a5601ac2f1732c9"
format_stamp: "Formatted at 2026-08-06 03:49:51 on dist-test-slave-6ntk"
server_key: "057eb8121ad3c406a7c4008f8251437d"
server_key_iv: "0067d601f4f89d4feec5d3d1d9fa3f33"
server_key_version: "encryptionkey@0"
I20260806 03:49:51.025744 1924 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:49:51.040786 1924 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:49:51.043246 1924 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:49:51.043424 1924 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:49:51.043696 1924 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 03:49:51.044047 1924 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 03:49:51.044106 1924 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 03:49:51.044152 1924 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 03:49:51.044175 1924 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 03:49:51.050267 1924 rpc_server.cc:307] RPC server started. Bound to: 127.11.253.193:42305
I20260806 03:49:51.050318 2083 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.253.193:42305 every 8 connection(s)
I20260806 03:49:51.050639 1924 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/minicluster-data/ts-0/data/info.pb
Aug 06 03:49:51 dist-test-slave-6ntk krb5kdc[1775](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988191, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.253.193@KRBTEST.COM for kudu/127.11.253.254@KRBTEST.COM
I20260806 03:49:51.057910 12279 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskToVWRe/build/release/bin/kudu as pid 1924
I20260806 03:49:51.058020 12279 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/minicluster-data/ts-0/wal/instance
I20260806 03:49:51.058269 12279 external_mini_cluster.cc:1428] Setting key 2f54923830f9ee2c8dee2aa5a87b6957
I20260806 03:49:51.060329 1890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:49:51.051374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.253.193:40991 (local address 127.11.253.254:42435)
0806 03:49:51.051448 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0806 03:49:51.051450 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:49:51.051657 (+ 207us) server_negotiation.cc:407] Connection header received
0806 03:49:51.052133 (+ 476us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:49:51.052134 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:49:51.052144 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:49:51.052199 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:49:51.053121 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:49:51.053619 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:49:51.054634 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:49:51.054829 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:49:51.054892 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:49:51.057122 (+ 2230us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:49:51.057124 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:49:51.057125 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:49:51.057134 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:49:51.058898 (+ 1764us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:49:51.059320 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:49:51.059321 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:49:51.059322 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:49:51.059390 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:49:51.059654 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:49:51.059655 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:49:51.059655 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:49:51.059777 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:49:51.059857 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0806 03:49:51.060135 (+ 278us) server_negotiation.cc:299] Negotiation successful
0806 03:49:51.060233 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260806 03:49:51.060536 2088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:49:51.051325 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:42435 (local address 127.11.253.193:40991)
0806 03:49:51.051525 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0806 03:49:51.051569 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0806 03:49:51.052051 (+ 482us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:49:51.052258 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:49:51.052262 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:49:51.052843 (+ 581us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:49:51.053045 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:49:51.053050 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:49:51.053701 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:49:51.053702 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:49:51.054561 (+ 859us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:49:51.054564 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:49:51.054912 (+ 348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:49:51.054912 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:49:51.055011 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:49:51.055504 (+ 493us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:49:51.055513 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:49:51.057040 (+ 1527us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:49:51.058989 (+ 1949us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:49:51.058997 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:49:51.058999 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:49:51.059235 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:49:51.059464 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:49:51.059465 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:49:51.059465 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:49:51.059582 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:49:51.059847 (+ 265us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:49:51.059854 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:49:51.060000 (+ 146us) client_negotiation.cc:769] Sending connection context
0806 03:49:51.060111 (+ 111us) client_negotiation.cc:240] Negotiation successful
0806 03:49:51.060274 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":109,"threads_started":1}
I20260806 03:49:51.061138 2086 heartbeater.cc:347] Connected to a master server at 127.11.253.254:42435
I20260806 03:49:51.061250 2086 heartbeater.cc:464] Registering TS with master...
I20260806 03:49:51.061431 2086 heartbeater.cc:511] Master 127.11.253.254:42435 requested a full tablet report, sending...
I20260806 03:49:51.062050 1842 ts_manager.cc:194] Registered new tserver with Master: 3f2f80aff6824a9a9a5601ac2f1732c9 (127.11.253.193:42305)
I20260806 03:49:51.062647 1842 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.253.193@KRBTEST.COM'} at 127.11.253.193:40991
WARNING: no policy specified for kudu/127.11.253.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.253.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.253.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.194.keytab.
Entry for principal kudu/127.11.253.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.194.keytab.
I20260806 03:49:51.088955 12279 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskToVWRe/build/release/bin/kudu
/tmp/dist-test-taskToVWRe/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.253.194:0
--local_ip_for_outbound_sockets=127.11.253.194
--webserver_interface=127.11.253.194
--webserver_port=0
--tserver_master_addrs=127.11.253.254:42435
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.194.keytab
--principal=kudu/127.11.253.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.253.212:40037
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:49:51.174129 2092 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:49:51.174345 2092 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:49:51.174367 2092 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:49:51.175535 2092 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:49:51.175571 2092 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:49:51.175597 2092 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.253.194
I20260806 03:49:51.176972 2092 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.253.212:40037
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.194.keytab
--principal=kudu/127.11.253.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.253.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.253.194
--webserver_port=0
--tserver_master_addrs=127.11.253.254:42435
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2092
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.253.194
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:49:51.177163 2092 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:49:51.177351 2092 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:49:51.182678 2102 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:49:51.182732 2105 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:49:51.182700 2103 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:49:51.182943 2092 server_base.cc:1034] running on GCE node
I20260806 03:49:51.183257 2092 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:49:51.183521 2092 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:49:51.184661 2092 hybrid_clock.cc:648] HybridClock initialized: now 1785988191184639 us; error 52 us; skew 500 ppm
Aug 06 03:49:51 dist-test-slave-6ntk krb5kdc[1775](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988191, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.253.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:49:51.187381 2092 init.cc:377] Logged in from keytab as kudu/127.11.253.194@KRBTEST.COM (short username kudu)
I20260806 03:49:51.187863 2092 webserver.cc:460] Webserver started at http://127.11.253.194:39327/ using document root <none> and password file <none>
I20260806 03:49:51.188130 2092 fs_manager.cc:359] Metadata directory not provided
I20260806 03:49:51.188201 2092 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:49:51.188351 2092 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:49:51.189440 2092 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/minicluster-data/ts-1/data/instance:
uuid: "fa87c812931045b48ed020a086658673"
format_stamp: "Formatted at 2026-08-06 03:49:51 on dist-test-slave-6ntk"
server_key: "090c6966b5b8a8ba039f3e35b75630e7"
server_key_iv: "0498c893b2e7d2f6c591c3e1ceb79195"
server_key_version: "encryptionkey@0"
I20260806 03:49:51.189807 2092 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/minicluster-data/ts-1/wal/instance:
uuid: "fa87c812931045b48ed020a086658673"
format_stamp: "Formatted at 2026-08-06 03:49:51 on dist-test-slave-6ntk"
server_key: "090c6966b5b8a8ba039f3e35b75630e7"
server_key_iv: "0498c893b2e7d2f6c591c3e1ceb79195"
server_key_version: "encryptionkey@0"
I20260806 03:49:51.191296 2092 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 03:49:51.192014 2114 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:49:51.192194 2092 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 03:49:51.192303 2092 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/minicluster-data/ts-1/data,/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/minicluster-data/ts-1/wal
uuid: "fa87c812931045b48ed020a086658673"
format_stamp: "Formatted at 2026-08-06 03:49:51 on dist-test-slave-6ntk"
server_key: "090c6966b5b8a8ba039f3e35b75630e7"
server_key_iv: "0498c893b2e7d2f6c591c3e1ceb79195"
server_key_version: "encryptionkey@0"
I20260806 03:49:51.192378 2092 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:49:51.209159 2092 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:49:51.211766 2092 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:49:51.211967 2092 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:49:51.212225 2092 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 03:49:51.212632 2092 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 03:49:51.212702 2092 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 03:49:51.212746 2092 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 03:49:51.212774 2092 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 03:49:51.219053 2092 rpc_server.cc:307] RPC server started. Bound to: 127.11.253.194:34953
I20260806 03:49:51.219105 2268 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.253.194:34953 every 8 connection(s)
I20260806 03:49:51.219440 2092 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/minicluster-data/ts-1/data/info.pb
I20260806 03:49:51.224783 12279 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskToVWRe/build/release/bin/kudu as pid 2092
I20260806 03:49:51.224911 12279 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/minicluster-data/ts-1/wal/instance
I20260806 03:49:51.225148 12279 external_mini_cluster.cc:1428] Setting key 2326434c9f92829029b5141f9d7c1acd
Aug 06 03:49:51 dist-test-slave-6ntk krb5kdc[1775](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988191, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.253.194@KRBTEST.COM for kudu/127.11.253.254@KRBTEST.COM
I20260806 03:49:51.229472 1890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:49:51.220179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.253.194:50903 (local address 127.11.253.254:42435)
0806 03:49:51.220250 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0806 03:49:51.220253 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 03:49:51.220490 (+ 237us) server_negotiation.cc:407] Connection header received
0806 03:49:51.220965 (+ 475us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:49:51.220965 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:49:51.220975 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:49:51.221034 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:49:51.222077 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:49:51.222578 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:49:51.223540 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:49:51.223725 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:49:51.223775 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:49:51.226279 (+ 2504us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:49:51.226281 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:49:51.226281 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:49:51.226290 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:49:51.227899 (+ 1609us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:49:51.228465 (+ 566us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:49:51.228467 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:49:51.228468 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:49:51.228538 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:49:51.228825 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:49:51.228826 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:49:51.228826 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:49:51.228935 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:49:51.228988 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0806 03:49:51.229323 (+ 335us) server_negotiation.cc:299] Negotiation successful
0806 03:49:51.229407 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260806 03:49:51.229717 2273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:49:51.220173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:42435 (local address 127.11.253.194:50903)
0806 03:49:51.220342 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0806 03:49:51.220396 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0806 03:49:51.220879 (+ 483us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:49:51.221088 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:49:51.221091 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:49:51.221702 (+ 611us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:49:51.221988 (+ 286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:49:51.221995 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:49:51.222659 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:49:51.222660 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:49:51.223462 (+ 802us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:49:51.223464 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:49:51.223798 (+ 334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:49:51.223799 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:49:51.223906 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:49:51.224411 (+ 505us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:49:51.224419 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:49:51.226185 (+ 1766us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:49:51.227995 (+ 1810us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:49:51.228002 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:49:51.228004 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:49:51.228326 (+ 322us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:49:51.228605 (+ 279us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:49:51.228606 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:49:51.228606 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:49:51.228756 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:49:51.229002 (+ 246us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:49:51.229007 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:49:51.229183 (+ 176us) client_negotiation.cc:769] Sending connection context
0806 03:49:51.229309 (+ 126us) client_negotiation.cc:240] Negotiation successful
0806 03:49:51.229442 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":90,"threads_started":1}
I20260806 03:49:51.230357 2269 heartbeater.cc:347] Connected to a master server at 127.11.253.254:42435
I20260806 03:49:51.230446 2269 heartbeater.cc:464] Registering TS with master...
I20260806 03:49:51.230641 2269 heartbeater.cc:511] Master 127.11.253.254:42435 requested a full tablet report, sending...
I20260806 03:49:51.231046 1842 ts_manager.cc:194] Registered new tserver with Master: fa87c812931045b48ed020a086658673 (127.11.253.194:34953)
I20260806 03:49:51.231437 1842 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.253.194@KRBTEST.COM'} at 127.11.253.194:50903
WARNING: no policy specified for kudu/127.11.253.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.253.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.253.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.195.keytab.
Entry for principal kudu/127.11.253.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.195.keytab.
I20260806 03:49:51.257005 12279 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskToVWRe/build/release/bin/kudu
/tmp/dist-test-taskToVWRe/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.253.195:0
--local_ip_for_outbound_sockets=127.11.253.195
--webserver_interface=127.11.253.195
--webserver_port=0
--tserver_master_addrs=127.11.253.254:42435
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.195.keytab
--principal=kudu/127.11.253.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.253.212:40037
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:49:51.343456 2278 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:49:51.343675 2278 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:49:51.343698 2278 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:49:51.344805 2278 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:49:51.344841 2278 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:49:51.344867 2278 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.253.195
I20260806 03:49:51.346370 2278 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.253.212:40037
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.195.keytab
--principal=kudu/127.11.253.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.253.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.253.195
--webserver_port=0
--tserver_master_addrs=127.11.253.254:42435
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2278
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.253.195
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:49:51.346570 2278 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:49:51.346755 2278 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:49:51.352043 2286 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:49:51.352128 2278 server_base.cc:1034] running on GCE node
W20260806 03:49:51.352049 2290 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:49:51.352049 2285 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:49:51.352478 2278 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:49:51.352730 2278 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:49:51.353922 2278 hybrid_clock.cc:648] HybridClock initialized: now 1785988191353893 us; error 45 us; skew 500 ppm
Aug 06 03:49:51 dist-test-slave-6ntk krb5kdc[1775](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988191, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.253.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:49:51.356606 2278 init.cc:377] Logged in from keytab as kudu/127.11.253.195@KRBTEST.COM (short username kudu)
I20260806 03:49:51.357132 2278 webserver.cc:460] Webserver started at http://127.11.253.195:43257/ using document root <none> and password file <none>
I20260806 03:49:51.357386 2278 fs_manager.cc:359] Metadata directory not provided
I20260806 03:49:51.357475 2278 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:49:51.357647 2278 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:49:51.358835 2278 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/minicluster-data/ts-2/data/instance:
uuid: "879486408a1b4379b9154994c41e45e9"
format_stamp: "Formatted at 2026-08-06 03:49:51 on dist-test-slave-6ntk"
server_key: "29100263602bca389eb0279894831131"
server_key_iv: "e8d05ae3aaccbc0c2abdd6af0c30829a"
server_key_version: "encryptionkey@0"
I20260806 03:49:51.359148 2278 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/minicluster-data/ts-2/wal/instance:
uuid: "879486408a1b4379b9154994c41e45e9"
format_stamp: "Formatted at 2026-08-06 03:49:51 on dist-test-slave-6ntk"
server_key: "29100263602bca389eb0279894831131"
server_key_iv: "e8d05ae3aaccbc0c2abdd6af0c30829a"
server_key_version: "encryptionkey@0"
I20260806 03:49:51.360448 2278 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 03:49:51.361151 2300 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:49:51.361320 2278 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 03:49:51.361415 2278 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/minicluster-data/ts-2/data,/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/minicluster-data/ts-2/wal
uuid: "879486408a1b4379b9154994c41e45e9"
format_stamp: "Formatted at 2026-08-06 03:49:51 on dist-test-slave-6ntk"
server_key: "29100263602bca389eb0279894831131"
server_key_iv: "e8d05ae3aaccbc0c2abdd6af0c30829a"
server_key_version: "encryptionkey@0"
I20260806 03:49:51.361474 2278 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:49:51.368070 2278 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:49:51.369814 2278 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:49:51.369998 2278 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:49:51.370190 2278 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 03:49:51.370451 2278 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 03:49:51.370481 2278 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 03:49:51.370502 2278 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 03:49:51.370514 2278 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 03:49:51.376221 2278 rpc_server.cc:307] RPC server started. Bound to: 127.11.253.195:34471
I20260806 03:49:51.376298 2460 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.253.195:34471 every 8 connection(s)
I20260806 03:49:51.376631 2278 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/minicluster-data/ts-2/data/info.pb
Aug 06 03:49:51 dist-test-slave-6ntk krb5kdc[1775](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988191, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.253.195@KRBTEST.COM for kudu/127.11.253.254@KRBTEST.COM
I20260806 03:49:51.383169 12279 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskToVWRe/build/release/bin/kudu as pid 2278
I20260806 03:49:51.383255 12279 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/minicluster-data/ts-2/wal/instance
I20260806 03:49:51.383395 12279 external_mini_cluster.cc:1428] Setting key 033a28494a01e012b49a0db2bea93b1b
I20260806 03:49:51.386785 1890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:49:51.377416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.253.195:59503 (local address 127.11.253.254:42435)
0806 03:49:51.377480 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0806 03:49:51.377482 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:49:51.377650 (+ 168us) server_negotiation.cc:407] Connection header received
0806 03:49:51.378315 (+ 665us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:49:51.378317 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:49:51.378333 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:49:51.378400 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:49:51.379361 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:49:51.379875 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:49:51.380928 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:49:51.381131 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:49:51.381209 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:49:51.383886 (+ 2677us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:49:51.383888 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:49:51.383889 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:49:51.383898 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:49:51.385345 (+ 1447us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:49:51.385769 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:49:51.385771 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:49:51.385771 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:49:51.385832 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:49:51.386135 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:49:51.386135 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:49:51.386136 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:49:51.386257 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:49:51.386335 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0806 03:49:51.386612 (+ 277us) server_negotiation.cc:299] Negotiation successful
0806 03:49:51.386692 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260806 03:49:51.386989 2463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:49:51.377361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:42435 (local address 127.11.253.195:59503)
0806 03:49:51.377521 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0806 03:49:51.377565 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0806 03:49:51.378179 (+ 614us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:49:51.378469 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:49:51.378472 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:49:51.379058 (+ 586us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:49:51.379255 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:49:51.379260 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:49:51.379975 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:49:51.379976 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:49:51.380821 (+ 845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:49:51.380823 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:49:51.381217 (+ 394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:49:51.381218 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:49:51.381343 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:49:51.381923 (+ 580us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:49:51.381931 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:49:51.383773 (+ 1842us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:49:51.385452 (+ 1679us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:49:51.385460 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:49:51.385462 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:49:51.385690 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:49:51.385947 (+ 257us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:49:51.385948 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:49:51.385949 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:49:51.386059 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:49:51.386326 (+ 267us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:49:51.386333 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:49:51.386473 (+ 140us) client_negotiation.cc:769] Sending connection context
0806 03:49:51.386577 (+ 104us) client_negotiation.cc:240] Negotiation successful
0806 03:49:51.386738 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":67,"threads_started":1}
I20260806 03:49:51.387452 2461 heartbeater.cc:347] Connected to a master server at 127.11.253.254:42435
I20260806 03:49:51.387547 2461 heartbeater.cc:464] Registering TS with master...
I20260806 03:49:51.387688 2461 heartbeater.cc:511] Master 127.11.253.254:42435 requested a full tablet report, sending...
I20260806 03:49:51.388100 1842 ts_manager.cc:194] Registered new tserver with Master: 879486408a1b4379b9154994c41e45e9 (127.11.253.195:34471)
I20260806 03:49:51.388441 1842 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.253.195@KRBTEST.COM'} at 127.11.253.195:59503
I20260806 03:49:51.395040 12279 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 03:49:51.402051 1890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:49:51.396353 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41584 (local address 127.11.253.254:42435)
0806 03:49:51.396435 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0806 03:49:51.396436 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 03:49:51.396546 (+ 110us) server_negotiation.cc:407] Connection header received
0806 03:49:51.396598 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:49:51.396599 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:49:51.396607 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:49:51.396647 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:49:51.397249 (+ 602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:49:51.397723 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:49:51.398463 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:49:51.398640 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:49:51.398699 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:49:51.399753 (+ 1054us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:49:51.399754 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:49:51.399755 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:49:51.399760 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:49:51.400951 (+ 1191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:49:51.401252 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:49:51.401253 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:49:51.401253 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:49:51.401309 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:49:51.401483 (+ 174us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:49:51.401483 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:49:51.401483 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:49:51.401617 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:49:51.401692 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0806 03:49:51.401852 (+ 160us) server_negotiation.cc:299] Negotiation successful
0806 03:49:51.401984 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260806 03:49:51.403541 1842 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41584:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 03:49:51.405055 1842 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 03:49:51.411980 2478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:49:51.407843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38308 (local address 127.11.253.195:34471)
0806 03:49:51.408132 (+ 289us) server_negotiation.cc:206] Beginning negotiation
0806 03:49:51.408136 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 03:49:51.408350 (+ 214us) server_negotiation.cc:407] Connection header received
0806 03:49:51.408427 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:49:51.408430 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:49:51.408445 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:49:51.408528 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 03:49:51.408606 (+ 78us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:49:51.409713 (+ 1107us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:49:51.411070 (+ 1357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:49:51.411570 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:49:51.411664 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:49:51.411728 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0806 03:49:51.411849 (+ 121us) server_negotiation.cc:299] Negotiation successful
0806 03:49:51.411898 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":251,"thread_start_us":115,"threads_started":1}
I20260806 03:49:51.411968 2475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:49:51.407606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.195:34471 (local address 127.0.0.1:38308)
0806 03:49:51.408237 (+ 631us) negotiation.cc:107] Waiting for socket to connect
0806 03:49:51.408246 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 03:49:51.408289 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:49:51.408523 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:49:51.408524 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:49:51.408527 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 03:49:51.408560 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:49:51.408561 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:49:51.409829 (+ 1268us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:49:51.409830 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:49:51.410983 (+ 1153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:49:51.410985 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:49:51.411661 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:49:51.411663 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:49:51.411742 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:49:51.411758 (+ 16us) client_negotiation.cc:769] Sending connection context
0806 03:49:51.411823 (+ 65us) client_negotiation.cc:240] Negotiation successful
0806 03:49:51.411881 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":614,"mutex_wait_us":35,"thread_start_us":94,"threads_started":1}
I20260806 03:49:51.412088 2479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:49:51.407876 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42624 (local address 127.11.253.194:34953)
0806 03:49:51.408040 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0806 03:49:51.408044 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 03:49:51.408143 (+ 99us) server_negotiation.cc:407] Connection header received
0806 03:49:51.408222 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:49:51.408225 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:49:51.408243 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:49:51.408325 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 03:49:51.408566 (+ 241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:49:51.409713 (+ 1147us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:49:51.410958 (+ 1245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:49:51.411656 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:49:51.411697 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:49:51.411739 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0806 03:49:51.411964 (+ 225us) server_negotiation.cc:299] Negotiation successful
0806 03:49:51.412009 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":85,"threads_started":1}
I20260806 03:49:51.412181 2476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:49:51.407696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59394 (local address 127.11.253.193:42305)
0806 03:49:51.407886 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0806 03:49:51.407890 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 03:49:51.408121 (+ 231us) server_negotiation.cc:407] Connection header received
0806 03:49:51.408189 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:49:51.408191 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:49:51.408209 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:49:51.408279 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 03:49:51.408640 (+ 361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:49:51.409327 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:49:51.410837 (+ 1510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:49:51.411605 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:49:51.411682 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:49:51.411740 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0806 03:49:51.412036 (+ 296us) server_negotiation.cc:299] Negotiation successful
0806 03:49:51.412083 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149,"thread_start_us":88,"threads_started":1}
I20260806 03:49:51.412328 2474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:49:51.407606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.194:34953 (local address 127.0.0.1:42624)
0806 03:49:51.408051 (+ 445us) negotiation.cc:107] Waiting for socket to connect
0806 03:49:51.408062 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 03:49:51.408149 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:49:51.408415 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:49:51.408417 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:49:51.408419 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 03:49:51.408479 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:49:51.408481 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:49:51.409842 (+ 1361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:49:51.409843 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:49:51.410847 (+ 1004us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:49:51.410850 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:49:51.411796 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:49:51.411797 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:49:51.411851 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:49:51.411859 (+ 8us) client_negotiation.cc:769] Sending connection context
0806 03:49:51.412207 (+ 348us) client_negotiation.cc:240] Negotiation successful
0806 03:49:51.412254 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":424,"mutex_wait_us":3,"thread_start_us":90,"threads_started":1}
I20260806 03:49:51.412489 2473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:49:51.407574 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.193:42305 (local address 127.0.0.1:59394)
0806 03:49:51.407903 (+ 329us) negotiation.cc:107] Waiting for socket to connect
0806 03:49:51.407947 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0806 03:49:51.408025 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:49:51.408377 (+ 352us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:49:51.408380 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:49:51.408383 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 03:49:51.408456 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:49:51.408459 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:49:51.409448 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:49:51.409450 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:49:51.410727 (+ 1277us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:49:51.410730 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:49:51.411777 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:49:51.411778 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:49:51.411851 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:49:51.411857 (+ 6us) client_negotiation.cc:769] Sending connection context
0806 03:49:51.411907 (+ 50us) client_negotiation.cc:240] Negotiation successful
0806 03:49:51.411959 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":294,"thread_start_us":73,"threads_started":1}
I20260806 03:49:51.413105 2179 tablet_service.cc:1467] Processing CreateTablet for tablet 4c8bb61adcde46d3860a2e83cc3a0470 (DEFAULT_TABLE table=test-table [id=c9cb2160fb03459eb091c67035a781df]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 03:49:51.413147 2373 tablet_service.cc:1467] Processing CreateTablet for tablet 4c8bb61adcde46d3860a2e83cc3a0470 (DEFAULT_TABLE table=test-table [id=c9cb2160fb03459eb091c67035a781df]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 03:49:51.413301 2008 tablet_service.cc:1467] Processing CreateTablet for tablet 4c8bb61adcde46d3860a2e83cc3a0470 (DEFAULT_TABLE table=test-table [id=c9cb2160fb03459eb091c67035a781df]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 03:49:51.413501 2373 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4c8bb61adcde46d3860a2e83cc3a0470. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:49:51.413515 2179 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4c8bb61adcde46d3860a2e83cc3a0470. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:49:51.413625 2008 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4c8bb61adcde46d3860a2e83cc3a0470. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:49:51.416196 2480 tablet_bootstrap.cc:492] T 4c8bb61adcde46d3860a2e83cc3a0470 P fa87c812931045b48ed020a086658673: Bootstrap starting.
I20260806 03:49:51.416893 2480 tablet_bootstrap.cc:654] T 4c8bb61adcde46d3860a2e83cc3a0470 P fa87c812931045b48ed020a086658673: Neither blocks nor log segments found. Creating new log.
I20260806 03:49:51.416874 2483 tablet_bootstrap.cc:492] T 4c8bb61adcde46d3860a2e83cc3a0470 P 3f2f80aff6824a9a9a5601ac2f1732c9: Bootstrap starting.
I20260806 03:49:51.417135 2485 tablet_bootstrap.cc:492] T 4c8bb61adcde46d3860a2e83cc3a0470 P 879486408a1b4379b9154994c41e45e9: Bootstrap starting.
I20260806 03:49:51.417573 2483 tablet_bootstrap.cc:654] T 4c8bb61adcde46d3860a2e83cc3a0470 P 3f2f80aff6824a9a9a5601ac2f1732c9: Neither blocks nor log segments found. Creating new log.
I20260806 03:49:51.417584 2480 log.cc:824] T 4c8bb61adcde46d3860a2e83cc3a0470 P fa87c812931045b48ed020a086658673: Log is configured to *not* fsync() on all Append() calls
I20260806 03:49:51.417742 2485 tablet_bootstrap.cc:654] T 4c8bb61adcde46d3860a2e83cc3a0470 P 879486408a1b4379b9154994c41e45e9: Neither blocks nor log segments found. Creating new log.
I20260806 03:49:51.418149 2483 log.cc:824] T 4c8bb61adcde46d3860a2e83cc3a0470 P 3f2f80aff6824a9a9a5601ac2f1732c9: Log is configured to *not* fsync() on all Append() calls
I20260806 03:49:51.418416 2485 log.cc:824] T 4c8bb61adcde46d3860a2e83cc3a0470 P 879486408a1b4379b9154994c41e45e9: Log is configured to *not* fsync() on all Append() calls
I20260806 03:49:51.418622 2480 tablet_bootstrap.cc:492] T 4c8bb61adcde46d3860a2e83cc3a0470 P fa87c812931045b48ed020a086658673: No bootstrap required, opened a new log
I20260806 03:49:51.418700 2480 ts_tablet_manager.cc:1397] T 4c8bb61adcde46d3860a2e83cc3a0470 P fa87c812931045b48ed020a086658673: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 03:49:51.419194 2483 tablet_bootstrap.cc:492] T 4c8bb61adcde46d3860a2e83cc3a0470 P 3f2f80aff6824a9a9a5601ac2f1732c9: No bootstrap required, opened a new log
I20260806 03:49:51.419268 2483 ts_tablet_manager.cc:1397] T 4c8bb61adcde46d3860a2e83cc3a0470 P 3f2f80aff6824a9a9a5601ac2f1732c9: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 03:49:51.420212 2480 raft_consensus.cc:348] T 4c8bb61adcde46d3860a2e83cc3a0470 P fa87c812931045b48ed020a086658673 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f2f80aff6824a9a9a5601ac2f1732c9" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 42305 } } peers { permanent_uuid: "879486408a1b4379b9154994c41e45e9" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 34471 } } peers { permanent_uuid: "fa87c812931045b48ed020a086658673" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 34953 } }
I20260806 03:49:51.420353 2480 raft_consensus.cc:374] T 4c8bb61adcde46d3860a2e83cc3a0470 P fa87c812931045b48ed020a086658673 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:49:51.420375 2480 raft_consensus.cc:729] T 4c8bb61adcde46d3860a2e83cc3a0470 P fa87c812931045b48ed020a086658673 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa87c812931045b48ed020a086658673, State: Initialized, Role: FOLLOWER
I20260806 03:49:51.420518 2480 consensus_queue.cc:260] T 4c8bb61adcde46d3860a2e83cc3a0470 P fa87c812931045b48ed020a086658673 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f2f80aff6824a9a9a5601ac2f1732c9" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 42305 } } peers { permanent_uuid: "879486408a1b4379b9154994c41e45e9" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 34471 } } peers { permanent_uuid: "fa87c812931045b48ed020a086658673" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 34953 } }
I20260806 03:49:51.420842 2480 ts_tablet_manager.cc:1428] T 4c8bb61adcde46d3860a2e83cc3a0470 P fa87c812931045b48ed020a086658673: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 03:49:51.420874 2269 heartbeater.cc:503] Master 127.11.253.254:42435 was elected leader, sending a full tablet report...
I20260806 03:49:51.420826 2483 raft_consensus.cc:348] T 4c8bb61adcde46d3860a2e83cc3a0470 P 3f2f80aff6824a9a9a5601ac2f1732c9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f2f80aff6824a9a9a5601ac2f1732c9" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 42305 } } peers { permanent_uuid: "879486408a1b4379b9154994c41e45e9" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 34471 } } peers { permanent_uuid: "fa87c812931045b48ed020a086658673" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 34953 } }
I20260806 03:49:51.420992 2483 raft_consensus.cc:374] T 4c8bb61adcde46d3860a2e83cc3a0470 P 3f2f80aff6824a9a9a5601ac2f1732c9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:49:51.421036 2483 raft_consensus.cc:729] T 4c8bb61adcde46d3860a2e83cc3a0470 P 3f2f80aff6824a9a9a5601ac2f1732c9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f2f80aff6824a9a9a5601ac2f1732c9, State: Initialized, Role: FOLLOWER
I20260806 03:49:51.421146 2483 consensus_queue.cc:260] T 4c8bb61adcde46d3860a2e83cc3a0470 P 3f2f80aff6824a9a9a5601ac2f1732c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f2f80aff6824a9a9a5601ac2f1732c9" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 42305 } } peers { permanent_uuid: "879486408a1b4379b9154994c41e45e9" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 34471 } } peers { permanent_uuid: "fa87c812931045b48ed020a086658673" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 34953 } }
I20260806 03:49:51.421453 2483 ts_tablet_manager.cc:1428] T 4c8bb61adcde46d3860a2e83cc3a0470 P 3f2f80aff6824a9a9a5601ac2f1732c9: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 03:49:51.421453 2086 heartbeater.cc:503] Master 127.11.253.254:42435 was elected leader, sending a full tablet report...
I20260806 03:49:51.422008 2485 tablet_bootstrap.cc:492] T 4c8bb61adcde46d3860a2e83cc3a0470 P 879486408a1b4379b9154994c41e45e9: No bootstrap required, opened a new log
I20260806 03:49:51.422078 2485 ts_tablet_manager.cc:1397] T 4c8bb61adcde46d3860a2e83cc3a0470 P 879486408a1b4379b9154994c41e45e9: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.002s
I20260806 03:49:51.423645 2485 raft_consensus.cc:348] T 4c8bb61adcde46d3860a2e83cc3a0470 P 879486408a1b4379b9154994c41e45e9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f2f80aff6824a9a9a5601ac2f1732c9" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 42305 } } peers { permanent_uuid: "879486408a1b4379b9154994c41e45e9" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 34471 } } peers { permanent_uuid: "fa87c812931045b48ed020a086658673" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 34953 } }
I20260806 03:49:51.423765 2485 raft_consensus.cc:374] T 4c8bb61adcde46d3860a2e83cc3a0470 P 879486408a1b4379b9154994c41e45e9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:49:51.423784 2485 raft_consensus.cc:729] T 4c8bb61adcde46d3860a2e83cc3a0470 P 879486408a1b4379b9154994c41e45e9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 879486408a1b4379b9154994c41e45e9, State: Initialized, Role: FOLLOWER
I20260806 03:49:51.423897 2485 consensus_queue.cc:260] T 4c8bb61adcde46d3860a2e83cc3a0470 P 879486408a1b4379b9154994c41e45e9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f2f80aff6824a9a9a5601ac2f1732c9" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 42305 } } peers { permanent_uuid: "879486408a1b4379b9154994c41e45e9" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 34471 } } peers { permanent_uuid: "fa87c812931045b48ed020a086658673" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 34953 } }
I20260806 03:49:51.424134 2485 ts_tablet_manager.cc:1428] T 4c8bb61adcde46d3860a2e83cc3a0470 P 879486408a1b4379b9154994c41e45e9: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260806 03:49:51.424204 2461 heartbeater.cc:503] Master 127.11.253.254:42435 was elected leader, sending a full tablet report...
W20260806 03:49:51.470098 2271 tablet.cc:2369] T 4c8bb61adcde46d3860a2e83cc3a0470 P fa87c812931045b48ed020a086658673: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 03:49:51.471642 2494 raft_consensus.cc:482] T 4c8bb61adcde46d3860a2e83cc3a0470 P 3f2f80aff6824a9a9a5601ac2f1732c9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 03:49:51.471750 2494 raft_consensus.cc:504] T 4c8bb61adcde46d3860a2e83cc3a0470 P 3f2f80aff6824a9a9a5601ac2f1732c9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f2f80aff6824a9a9a5601ac2f1732c9" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 42305 } } peers { permanent_uuid: "879486408a1b4379b9154994c41e45e9" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 34471 } } peers { permanent_uuid: "fa87c812931045b48ed020a086658673" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 34953 } }
I20260806 03:49:51.472157 2494 leader_election.cc:288] T 4c8bb61adcde46d3860a2e83cc3a0470 P 3f2f80aff6824a9a9a5601ac2f1732c9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 879486408a1b4379b9154994c41e45e9 (127.11.253.195:34471), fa87c812931045b48ed020a086658673 (127.11.253.194:34953)
I20260806 03:49:51.475052 2478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:49:51.472500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.253.193:51351 (local address 127.11.253.195:34471)
0806 03:49:51.472567 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0806 03:49:51.472569 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:49:51.472586 (+ 17us) server_negotiation.cc:407] Connection header received
0806 03:49:51.472608 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:49:51.472608 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:49:51.472617 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:49:51.472687 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 03:49:51.472857 (+ 170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:49:51.473337 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:49:51.474330 (+ 993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:49:51.474701 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:49:51.474787 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:49:51.474797 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0806 03:49:51.474944 (+ 147us) server_negotiation.cc:299] Negotiation successful
0806 03:49:51.474981 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260806 03:49:51.475085 2088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:49:51.472362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.195:34471 (local address 127.11.253.193:51351)
0806 03:49:51.472416 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0806 03:49:51.472429 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 03:49:51.472472 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:49:51.472694 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:49:51.472695 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:49:51.472697 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 03:49:51.472779 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:49:51.472780 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:49:51.473443 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:49:51.473444 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:49:51.474240 (+ 796us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:49:51.474242 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:49:51.474787 (+ 545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:49:51.474788 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:49:51.474868 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:49:51.474871 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 03:49:51.474937 (+ 66us) client_negotiation.cc:240] Negotiation successful
0806 03:49:51.474980 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260806 03:49:51.475122 2479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:49:51.472464 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.253.193:51949 (local address 127.11.253.194:34953)
0806 03:49:51.472514 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0806 03:49:51.472517 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 03:49:51.472672 (+ 155us) server_negotiation.cc:407] Connection header received
0806 03:49:51.472748 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:49:51.472749 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:49:51.472760 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:49:51.472828 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 03:49:51.472989 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:49:51.473449 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:49:51.474379 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:49:51.474755 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:49:51.474839 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:49:51.474848 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0806 03:49:51.475011 (+ 163us) server_negotiation.cc:299] Negotiation successful
0806 03:49:51.475049 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260806 03:49:51.475193 2496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:49:51.472364 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.194:34953 (local address 127.11.253.193:51949)
0806 03:49:51.472589 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0806 03:49:51.472598 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 03:49:51.472678 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:49:51.472833 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:49:51.472834 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:49:51.472835 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 03:49:51.472916 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:49:51.472917 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:49:51.473534 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:49:51.473535 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:49:51.474288 (+ 753us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:49:51.474290 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:49:51.474840 (+ 550us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:49:51.474841 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:49:51.474919 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:49:51.474922 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 03:49:51.474993 (+ 71us) client_negotiation.cc:240] Negotiation successful
0806 03:49:51.475031 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":209,"mutex_wait_us":13,"thread_start_us":147,"threads_started":1}
I20260806 03:49:51.475507 2400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4c8bb61adcde46d3860a2e83cc3a0470" candidate_uuid: "3f2f80aff6824a9a9a5601ac2f1732c9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "879486408a1b4379b9154994c41e45e9" is_pre_election: true
I20260806 03:49:51.475584 2211 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4c8bb61adcde46d3860a2e83cc3a0470" candidate_uuid: "3f2f80aff6824a9a9a5601ac2f1732c9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa87c812931045b48ed020a086658673" is_pre_election: true
I20260806 03:49:51.475680 2400 raft_consensus.cc:2454] T 4c8bb61adcde46d3860a2e83cc3a0470 P 879486408a1b4379b9154994c41e45e9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f2f80aff6824a9a9a5601ac2f1732c9 in term 0.
I20260806 03:49:51.475729 2211 raft_consensus.cc:2454] T 4c8bb61adcde46d3860a2e83cc3a0470 P fa87c812931045b48ed020a086658673 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f2f80aff6824a9a9a5601ac2f1732c9 in term 0.
I20260806 03:49:51.475961 1944 leader_election.cc:302] T 4c8bb61adcde46d3860a2e83cc3a0470 P 3f2f80aff6824a9a9a5601ac2f1732c9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f2f80aff6824a9a9a5601ac2f1732c9, fa87c812931045b48ed020a086658673; no voters:
I20260806 03:49:51.476133 2494 raft_consensus.cc:2781] T 4c8bb61adcde46d3860a2e83cc3a0470 P 3f2f80aff6824a9a9a5601ac2f1732c9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 03:49:51.476192 2494 raft_consensus.cc:482] T 4c8bb61adcde46d3860a2e83cc3a0470 P 3f2f80aff6824a9a9a5601ac2f1732c9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 03:49:51.476225 2494 raft_consensus.cc:3037] T 4c8bb61adcde46d3860a2e83cc3a0470 P 3f2f80aff6824a9a9a5601ac2f1732c9 [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:49:51.476971 2494 raft_consensus.cc:504] T 4c8bb61adcde46d3860a2e83cc3a0470 P 3f2f80aff6824a9a9a5601ac2f1732c9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f2f80aff6824a9a9a5601ac2f1732c9" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 42305 } } peers { permanent_uuid: "879486408a1b4379b9154994c41e45e9" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 34471 } } peers { permanent_uuid: "fa87c812931045b48ed020a086658673" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 34953 } }
I20260806 03:49:51.477119 2494 leader_election.cc:288] T 4c8bb61adcde46d3860a2e83cc3a0470 P 3f2f80aff6824a9a9a5601ac2f1732c9 [CANDIDATE]: Term 1 election: Requested vote from peers 879486408a1b4379b9154994c41e45e9 (127.11.253.195:34471), fa87c812931045b48ed020a086658673 (127.11.253.194:34953)
I20260806 03:49:51.477301 2400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4c8bb61adcde46d3860a2e83cc3a0470" candidate_uuid: "3f2f80aff6824a9a9a5601ac2f1732c9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "879486408a1b4379b9154994c41e45e9"
I20260806 03:49:51.477316 2211 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4c8bb61adcde46d3860a2e83cc3a0470" candidate_uuid: "3f2f80aff6824a9a9a5601ac2f1732c9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa87c812931045b48ed020a086658673"
I20260806 03:49:51.477401 2211 raft_consensus.cc:3037] T 4c8bb61adcde46d3860a2e83cc3a0470 P fa87c812931045b48ed020a086658673 [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:49:51.477397 2400 raft_consensus.cc:3037] T 4c8bb61adcde46d3860a2e83cc3a0470 P 879486408a1b4379b9154994c41e45e9 [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:49:51.478185 2211 raft_consensus.cc:2454] T 4c8bb61adcde46d3860a2e83cc3a0470 P fa87c812931045b48ed020a086658673 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f2f80aff6824a9a9a5601ac2f1732c9 in term 1.
I20260806 03:49:51.478257 2400 raft_consensus.cc:2454] T 4c8bb61adcde46d3860a2e83cc3a0470 P 879486408a1b4379b9154994c41e45e9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f2f80aff6824a9a9a5601ac2f1732c9 in term 1.
I20260806 03:49:51.478401 1944 leader_election.cc:302] T 4c8bb61adcde46d3860a2e83cc3a0470 P 3f2f80aff6824a9a9a5601ac2f1732c9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f2f80aff6824a9a9a5601ac2f1732c9, fa87c812931045b48ed020a086658673; no voters:
I20260806 03:49:51.478480 2494 raft_consensus.cc:2781] T 4c8bb61adcde46d3860a2e83cc3a0470 P 3f2f80aff6824a9a9a5601ac2f1732c9 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 03:49:51.478641 2494 raft_consensus.cc:686] T 4c8bb61adcde46d3860a2e83cc3a0470 P 3f2f80aff6824a9a9a5601ac2f1732c9 [term 1 LEADER]: Becoming Leader. State: Replica: 3f2f80aff6824a9a9a5601ac2f1732c9, State: Running, Role: LEADER
I20260806 03:49:51.478749 2494 consensus_queue.cc:237] T 4c8bb61adcde46d3860a2e83cc3a0470 P 3f2f80aff6824a9a9a5601ac2f1732c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f2f80aff6824a9a9a5601ac2f1732c9" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 42305 } } peers { permanent_uuid: "879486408a1b4379b9154994c41e45e9" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 34471 } } peers { permanent_uuid: "fa87c812931045b48ed020a086658673" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 34953 } }
I20260806 03:49:51.479628 1841 catalog_manager.cc:5495] T 4c8bb61adcde46d3860a2e83cc3a0470 P 3f2f80aff6824a9a9a5601ac2f1732c9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3f2f80aff6824a9a9a5601ac2f1732c9 (127.11.253.193). New cstate: current_term: 1 leader_uuid: "3f2f80aff6824a9a9a5601ac2f1732c9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f2f80aff6824a9a9a5601ac2f1732c9" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 42305 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "879486408a1b4379b9154994c41e45e9" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 34471 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fa87c812931045b48ed020a086658673" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 34953 } health_report { overall_health: UNKNOWN } } }
I20260806 03:49:51.488559 2476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:49:51.486175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59400 (local address 127.11.253.193:42305)
0806 03:49:51.486235 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0806 03:49:51.486237 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:49:51.486246 (+ 9us) server_negotiation.cc:407] Connection header received
0806 03:49:51.486327 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:49:51.486328 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:49:51.486353 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:49:51.486436 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 03:49:51.486585 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:49:51.487033 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:49:51.487783 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:49:51.487969 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:49:51.488041 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:49:51.488197 (+ 156us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 03:49:51.488294 (+ 97us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 03:49:51.488399 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0806 03:49:51.488416 (+ 17us) server_negotiation.cc:299] Negotiation successful
0806 03:49:51.488436 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260806 03:49:51.490314 2211 raft_consensus.cc:1261] T 4c8bb61adcde46d3860a2e83cc3a0470 P fa87c812931045b48ed020a086658673 [term 1 FOLLOWER]: Refusing update from remote peer 3f2f80aff6824a9a9a5601ac2f1732c9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 03:49:51.490348 2400 raft_consensus.cc:1261] T 4c8bb61adcde46d3860a2e83cc3a0470 P 879486408a1b4379b9154994c41e45e9 [term 1 FOLLOWER]: Refusing update from remote peer 3f2f80aff6824a9a9a5601ac2f1732c9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 03:49:51.490674 2497 consensus_queue.cc:1035] T 4c8bb61adcde46d3860a2e83cc3a0470 P 3f2f80aff6824a9a9a5601ac2f1732c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "879486408a1b4379b9154994c41e45e9" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 34471 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 03:49:51.490926 2494 consensus_queue.cc:1035] T 4c8bb61adcde46d3860a2e83cc3a0470 P 3f2f80aff6824a9a9a5601ac2f1732c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa87c812931045b48ed020a086658673" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 34953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 03:49:51.492736 2502 mvcc.cc:205] Tried to move back new op lower bound from 7315407632341364736 to 7315407632297545728. Current Snapshot: MvccSnapshot[applied={T|T < 7315407632341364736}]
I20260806 03:49:51.493418 2503 mvcc.cc:205] Tried to move back new op lower bound from 7315407632341364736 to 7315407632297545728. Current Snapshot: MvccSnapshot[applied={T|T < 7315407632341364736}]
I20260806 03:49:51.495054 1841 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41584:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 03:49:51.495146 1841 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41584:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 03:49:51.495935 1841 catalog_manager.cc:5782] T 00000000000000000000000000000000 P eba5da44f0f04fc6bdc7488532ffbd26: Sending DeleteTablet for 3 replicas of tablet 4c8bb61adcde46d3860a2e83cc3a0470
I20260806 03:49:51.496261 2373 tablet_service.cc:1514] Processing DeleteTablet for tablet 4c8bb61adcde46d3860a2e83cc3a0470 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 03:49:51 UTC) from {username='kudu', principal='kudu/127.11.253.254@KRBTEST.COM'} at 127.0.0.1:38308
I20260806 03:49:51.496340 2179 tablet_service.cc:1514] Processing DeleteTablet for tablet 4c8bb61adcde46d3860a2e83cc3a0470 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 03:49:51 UTC) from {username='kudu', principal='kudu/127.11.253.254@KRBTEST.COM'} at 127.0.0.1:42624
I20260806 03:49:51.496472 2511 tablet_replica.cc:330] T 4c8bb61adcde46d3860a2e83cc3a0470 P 879486408a1b4379b9154994c41e45e9: stopping tablet replica
I20260806 03:49:51.496573 2511 raft_consensus.cc:2229] T 4c8bb61adcde46d3860a2e83cc3a0470 P 879486408a1b4379b9154994c41e45e9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 03:49:51.496567 12279 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskToVWRe/build/release/bin/kudu with pid 1924
I20260806 03:49:51.496708 2511 raft_consensus.cc:2258] T 4c8bb61adcde46d3860a2e83cc3a0470 P 879486408a1b4379b9154994c41e45e9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 03:49:51.497174 2511 ts_tablet_manager.cc:1905] T 4c8bb61adcde46d3860a2e83cc3a0470 P 879486408a1b4379b9154994c41e45e9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 03:49:51.499976 2511 ts_tablet_manager.cc:1918] T 4c8bb61adcde46d3860a2e83cc3a0470 P 879486408a1b4379b9154994c41e45e9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 03:49:51.500044 2511 log.cc:1196] T 4c8bb61adcde46d3860a2e83cc3a0470 P 879486408a1b4379b9154994c41e45e9: Deleting WAL directory at /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/minicluster-data/ts-2/wal/wals/4c8bb61adcde46d3860a2e83cc3a0470
I20260806 03:49:51.500308 2511 ts_tablet_manager.cc:1939] T 4c8bb61adcde46d3860a2e83cc3a0470 P 879486408a1b4379b9154994c41e45e9: Deleting consensus metadata
I20260806 03:49:51.500763 1820 catalog_manager.cc:4841] TS 879486408a1b4379b9154994c41e45e9 (127.11.253.195:34471): tablet 4c8bb61adcde46d3860a2e83cc3a0470 (table test-table [id=c9cb2160fb03459eb091c67035a781df]) successfully deleted
I20260806 03:49:51.506877 2512 tablet_replica.cc:330] T 4c8bb61adcde46d3860a2e83cc3a0470 P fa87c812931045b48ed020a086658673: stopping tablet replica
I20260806 03:49:51.507018 2512 raft_consensus.cc:2229] T 4c8bb61adcde46d3860a2e83cc3a0470 P fa87c812931045b48ed020a086658673 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 03:49:51.507153 2512 raft_consensus.cc:2258] T 4c8bb61adcde46d3860a2e83cc3a0470 P fa87c812931045b48ed020a086658673 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 03:49:51.507593 2512 ts_tablet_manager.cc:1905] T 4c8bb61adcde46d3860a2e83cc3a0470 P fa87c812931045b48ed020a086658673: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 03:49:51.509382 2512 ts_tablet_manager.cc:1918] T 4c8bb61adcde46d3860a2e83cc3a0470 P fa87c812931045b48ed020a086658673: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 03:49:51.509502 2512 log.cc:1196] T 4c8bb61adcde46d3860a2e83cc3a0470 P fa87c812931045b48ed020a086658673: Deleting WAL directory at /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/minicluster-data/ts-1/wal/wals/4c8bb61adcde46d3860a2e83cc3a0470
I20260806 03:49:51.509781 2512 ts_tablet_manager.cc:1939] T 4c8bb61adcde46d3860a2e83cc3a0470 P fa87c812931045b48ed020a086658673: Deleting consensus metadata
I20260806 03:49:51.510318 1817 catalog_manager.cc:4841] TS fa87c812931045b48ed020a086658673 (127.11.253.194:34953): tablet 4c8bb61adcde46d3860a2e83cc3a0470 (table test-table [id=c9cb2160fb03459eb091c67035a781df]) successfully deleted
W20260806 03:49:51.512043 1819 connection.cc:535] client connection to 127.11.253.193:42305 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260806 03:49:51.512180 1819 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260806 03:49:51.512329 12279 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskToVWRe/build/release/bin/kudu with pid 2092
I20260806 03:49:51.516018 2473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:49:51.515817 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.193:42305 (local address 127.0.0.1:59408)
0806 03:49:51.515909 (+ 92us) negotiation.cc:107] Waiting for socket to connect
0806 03:49:51.515967 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.253.193:42305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260806 03:49:51.516193 1819 catalog_manager.cc:4584] TS 3f2f80aff6824a9a9a5601ac2f1732c9 (127.11.253.193:42305): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 4c8bb61adcde46d3860a2e83cc3a0470: Network error: Client connection negotiation failed: client connection to 127.11.253.193:42305: connect: Connection refused (error 111)
I20260806 03:49:51.521852 12279 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskToVWRe/build/release/bin/kudu with pid 2278
I20260806 03:49:51.532529 12279 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskToVWRe/build/release/bin/kudu with pid 1795
I20260806 03:50:11.544376 12279 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskToVWRe/build/release/bin/kudu
/tmp/dist-test-taskToVWRe/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.253.254:42435
--webserver_interface=127.11.253.254
--webserver_port=39741
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.254.keytab
--principal=kudu/127.11.253.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.253.212:40037
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.253.254:42435
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:50:11.629273 2515 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:50:11.629491 2515 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:50:11.629513 2515 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:50:11.630699 2515 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260806 03:50:11.630746 2515 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260806 03:50:11.630760 2515 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260806 03:50:11.630772 2515 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 03:50:11.630785 2515 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:50:11.630836 2515 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:50:11.630862 2515 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 03:50:11.630885 2515 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 03:50:11.632411 2515 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.253.212:40037
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.11.253.254:42435
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.254.keytab
--principal=kudu/127.11.253.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.253.254:42435
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.253.254
--webserver_port=39741
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2515
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:50:11.632668 2515 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:50:11.632933 2515 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:50:11.642926 2530 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:50:11.642971 2525 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:50:11.642992 2524 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:50:11.643040 2515 server_base.cc:1034] running on GCE node
I20260806 03:50:11.643481 2515 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:50:11.643867 2515 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:50:11.645047 2515 hybrid_clock.cc:648] HybridClock initialized: now 1785988211645020 us; error 38 us; skew 500 ppm
Aug 06 03:50:11 dist-test-slave-6ntk krb5kdc[1775](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988211, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.253.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:50:11.652575 2515 init.cc:377] Logged in from keytab as kudu/127.11.253.254@KRBTEST.COM (short username kudu)
I20260806 03:50:11.653002 2515 webserver.cc:460] Webserver started at http://127.11.253.254:39741/ using document root <none> and password file <none>
I20260806 03:50:11.653208 2515 fs_manager.cc:359] Metadata directory not provided
I20260806 03:50:11.653262 2515 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:50:11.654734 2515 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.000s
I20260806 03:50:11.655377 2545 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:50:11.655557 2515 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 03:50:11.655647 2515 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/minicluster-data/master-0/data,/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/minicluster-data/master-0/wal
uuid: "eba5da44f0f04fc6bdc7488532ffbd26"
format_stamp: "Formatted at 2026-08-06 03:49:50 on dist-test-slave-6ntk"
server_key: "1ca51766568e6569ae46743e4cc146f9"
server_key_iv: "28d770e821969572ffc0c2ff0cfbdb81"
server_key_version: "encryptionkey@0"
I20260806 03:50:11.656057 2515 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:50:11.666782 2515 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:50:11.668715 2515 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:50:11.668828 2515 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:50:11.672865 2515 rpc_server.cc:307] RPC server started. Bound to: 127.11.253.254:42435
I20260806 03:50:11.673173 2618 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.253.254:42435 every 8 connection(s)
I20260806 03:50:11.673300 2515 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/minicluster-data/master-0/data/info.pb
I20260806 03:50:11.674729 2619 sys_catalog.cc:263] Verifying existing consensus state
I20260806 03:50:11.675415 2619 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eba5da44f0f04fc6bdc7488532ffbd26: Bootstrap starting.
I20260806 03:50:11.680810 12279 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskToVWRe/build/release/bin/kudu as pid 2515
I20260806 03:50:11.681102 12279 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskToVWRe/build/release/bin/kudu
/tmp/dist-test-taskToVWRe/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.253.193:42305
--local_ip_for_outbound_sockets=127.11.253.193
--tserver_master_addrs=127.11.253.254:42435
--webserver_port=44441
--webserver_interface=127.11.253.193
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.193.keytab
--principal=kudu/127.11.253.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.253.212:40037
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 03:50:11.685070 2619 log.cc:824] T 00000000000000000000000000000000 P eba5da44f0f04fc6bdc7488532ffbd26: Log is configured to *not* fsync() on all Append() calls
I20260806 03:50:11.687134 2619 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eba5da44f0f04fc6bdc7488532ffbd26: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 03:50:11.687453 2619 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eba5da44f0f04fc6bdc7488532ffbd26: Bootstrap complete.
I20260806 03:50:11.689805 2619 raft_consensus.cc:348] T 00000000000000000000000000000000 P eba5da44f0f04fc6bdc7488532ffbd26 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eba5da44f0f04fc6bdc7488532ffbd26" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 42435 } }
I20260806 03:50:11.690115 2619 raft_consensus.cc:729] T 00000000000000000000000000000000 P eba5da44f0f04fc6bdc7488532ffbd26 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: eba5da44f0f04fc6bdc7488532ffbd26, State: Initialized, Role: FOLLOWER
I20260806 03:50:11.690280 2619 consensus_queue.cc:260] T 00000000000000000000000000000000 P eba5da44f0f04fc6bdc7488532ffbd26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eba5da44f0f04fc6bdc7488532ffbd26" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 42435 } }
I20260806 03:50:11.690339 2619 raft_consensus.cc:388] T 00000000000000000000000000000000 P eba5da44f0f04fc6bdc7488532ffbd26 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 03:50:11.690400 2619 raft_consensus.cc:482] T 00000000000000000000000000000000 P eba5da44f0f04fc6bdc7488532ffbd26 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 03:50:11.690459 2619 raft_consensus.cc:3037] T 00000000000000000000000000000000 P eba5da44f0f04fc6bdc7488532ffbd26 [term 1 FOLLOWER]: Advancing to term 2
I20260806 03:50:11.691632 2619 raft_consensus.cc:504] T 00000000000000000000000000000000 P eba5da44f0f04fc6bdc7488532ffbd26 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eba5da44f0f04fc6bdc7488532ffbd26" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 42435 } }
I20260806 03:50:11.691721 2619 leader_election.cc:302] T 00000000000000000000000000000000 P eba5da44f0f04fc6bdc7488532ffbd26 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: eba5da44f0f04fc6bdc7488532ffbd26; no voters:
I20260806 03:50:11.691937 2619 leader_election.cc:288] T 00000000000000000000000000000000 P eba5da44f0f04fc6bdc7488532ffbd26 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260806 03:50:11.691972 2624 raft_consensus.cc:2781] T 00000000000000000000000000000000 P eba5da44f0f04fc6bdc7488532ffbd26 [term 2 FOLLOWER]: Leader election won for term 2
I20260806 03:50:11.692094 2624 raft_consensus.cc:686] T 00000000000000000000000000000000 P eba5da44f0f04fc6bdc7488532ffbd26 [term 2 LEADER]: Becoming Leader. State: Replica: eba5da44f0f04fc6bdc7488532ffbd26, State: Running, Role: LEADER
I20260806 03:50:11.692183 2624 consensus_queue.cc:237] T 00000000000000000000000000000000 P eba5da44f0f04fc6bdc7488532ffbd26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eba5da44f0f04fc6bdc7488532ffbd26" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 42435 } }
I20260806 03:50:11.692209 2619 sys_catalog.cc:564] T 00000000000000000000000000000000 P eba5da44f0f04fc6bdc7488532ffbd26 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 03:50:11.692492 2625 sys_catalog.cc:455] T 00000000000000000000000000000000 P eba5da44f0f04fc6bdc7488532ffbd26 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "eba5da44f0f04fc6bdc7488532ffbd26" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eba5da44f0f04fc6bdc7488532ffbd26" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 42435 } } }
I20260806 03:50:11.692638 2625 sys_catalog.cc:458] T 00000000000000000000000000000000 P eba5da44f0f04fc6bdc7488532ffbd26 [sys.catalog]: This master's current role is: LEADER
I20260806 03:50:11.692902 2631 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 03:50:11.692924 2626 sys_catalog.cc:455] T 00000000000000000000000000000000 P eba5da44f0f04fc6bdc7488532ffbd26 [sys.catalog]: SysCatalogTable state changed. Reason: New leader eba5da44f0f04fc6bdc7488532ffbd26. Latest consensus state: current_term: 2 leader_uuid: "eba5da44f0f04fc6bdc7488532ffbd26" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eba5da44f0f04fc6bdc7488532ffbd26" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 42435 } } }
I20260806 03:50:11.693022 2626 sys_catalog.cc:458] T 00000000000000000000000000000000 P eba5da44f0f04fc6bdc7488532ffbd26 [sys.catalog]: This master's current role is: LEADER
I20260806 03:50:11.693771 2631 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 03:50:11.693895 2631 catalog_manager.cc:1259] Loaded cluster ID: 93da3e1b34ed419e8b9740addf6b9ad8
I20260806 03:50:11.693928 2631 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 03:50:11.695382 2631 catalog_manager.cc:1504] Loading token signing keys...
I20260806 03:50:11.695559 2631 catalog_manager.cc:5879] T 00000000000000000000000000000000 P eba5da44f0f04fc6bdc7488532ffbd26: Loaded TSK: 0
I20260806 03:50:11.706688 2631 catalog_manager.cc:5868] T 00000000000000000000000000000000 P eba5da44f0f04fc6bdc7488532ffbd26: Generated new TSK 1
I20260806 03:50:11.706784 2631 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 03:50:11.771376 2622 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:50:11.771577 2622 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:50:11.771600 2622 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:50:11.772804 2622 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:50:11.772850 2622 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:50:11.772878 2622 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.253.193
I20260806 03:50:11.774430 2622 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.253.212:40037
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.193.keytab
--principal=kudu/127.11.253.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.253.193:42305
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.253.193
--webserver_port=44441
--tserver_master_addrs=127.11.253.254:42435
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2622
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.253.193
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:50:11.774674 2622 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:50:11.774930 2622 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 03:50:11.780519 2622 server_base.cc:1034] running on GCE node
W20260806 03:50:11.780457 2658 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:50:11.780684 2660 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:50:11.780812 2655 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:50:11.781065 2622 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:50:11.781327 2622 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:50:11.782514 2622 hybrid_clock.cc:648] HybridClock initialized: now 1785988211782495 us; error 40 us; skew 500 ppm
Aug 06 03:50:11 dist-test-slave-6ntk krb5kdc[1775](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988211, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.253.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:50:11.785662 2622 init.cc:377] Logged in from keytab as kudu/127.11.253.193@KRBTEST.COM (short username kudu)
I20260806 03:50:11.786201 2622 webserver.cc:460] Webserver started at http://127.11.253.193:44441/ using document root <none> and password file <none>
I20260806 03:50:11.786514 2622 fs_manager.cc:359] Metadata directory not provided
I20260806 03:50:11.786598 2622 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:50:11.787994 2622 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 03:50:11.788702 2669 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:50:11.788895 2622 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 03:50:11.789007 2622 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/minicluster-data/ts-0/data,/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/minicluster-data/ts-0/wal
uuid: "3f2f80aff6824a9a9a5601ac2f1732c9"
format_stamp: "Formatted at 2026-08-06 03:49:51 on dist-test-slave-6ntk"
server_key: "057eb8121ad3c406a7c4008f8251437d"
server_key_iv: "0067d601f4f89d4feec5d3d1d9fa3f33"
server_key_version: "encryptionkey@0"
I20260806 03:50:11.789382 2622 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:50:11.807426 2622 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:50:11.809849 2622 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:50:11.810070 2622 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:50:11.810344 2622 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 03:50:11.810850 2682 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260806 03:50:11.812417 2622 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260806 03:50:11.812470 2622 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260806 03:50:11.812496 2622 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260806 03:50:11.813170 2622 ts_tablet_manager.cc:610] Registered 1 tablets
I20260806 03:50:11.813215 2622 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260806 03:50:11.813267 2682 tablet_bootstrap.cc:492] T 4c8bb61adcde46d3860a2e83cc3a0470 P 3f2f80aff6824a9a9a5601ac2f1732c9: Bootstrap starting.
I20260806 03:50:11.821173 2622 rpc_server.cc:307] RPC server started. Bound to: 127.11.253.193:42305
I20260806 03:50:11.821367 2823 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.253.193:42305 every 8 connection(s)
I20260806 03:50:11.821732 2622 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/minicluster-data/ts-0/data/info.pb
I20260806 03:50:11.825094 2682 log.cc:824] T 4c8bb61adcde46d3860a2e83cc3a0470 P 3f2f80aff6824a9a9a5601ac2f1732c9: Log is configured to *not* fsync() on all Append() calls
I20260806 03:50:11.827234 12279 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskToVWRe/build/release/bin/kudu as pid 2622
I20260806 03:50:11.827565 12279 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskToVWRe/build/release/bin/kudu
/tmp/dist-test-taskToVWRe/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.253.194:34953
--local_ip_for_outbound_sockets=127.11.253.194
--tserver_master_addrs=127.11.253.254:42435
--webserver_port=39327
--webserver_interface=127.11.253.194
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.194.keytab
--principal=kudu/127.11.253.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.253.212:40037
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 03:50:11.827661 2682 tablet_bootstrap.cc:492] T 4c8bb61adcde46d3860a2e83cc3a0470 P 3f2f80aff6824a9a9a5601ac2f1732c9: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 03:50:11.827934 2682 tablet_bootstrap.cc:492] T 4c8bb61adcde46d3860a2e83cc3a0470 P 3f2f80aff6824a9a9a5601ac2f1732c9: Bootstrap complete.
I20260806 03:50:11.829026 2682 ts_tablet_manager.cc:1397] T 4c8bb61adcde46d3860a2e83cc3a0470 P 3f2f80aff6824a9a9a5601ac2f1732c9: Time spent bootstrapping tablet: real 0.016s user 0.012s sys 0.000s
Aug 06 03:50:11 dist-test-slave-6ntk krb5kdc[1775](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988211, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.253.193@KRBTEST.COM for kudu/127.11.253.254@KRBTEST.COM
I20260806 03:50:11.830446 2682 raft_consensus.cc:348] T 4c8bb61adcde46d3860a2e83cc3a0470 P 3f2f80aff6824a9a9a5601ac2f1732c9 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f2f80aff6824a9a9a5601ac2f1732c9" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 42305 } } peers { permanent_uuid: "879486408a1b4379b9154994c41e45e9" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 34471 } } peers { permanent_uuid: "fa87c812931045b48ed020a086658673" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 34953 } }
I20260806 03:50:11.830837 2682 raft_consensus.cc:729] T 4c8bb61adcde46d3860a2e83cc3a0470 P 3f2f80aff6824a9a9a5601ac2f1732c9 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f2f80aff6824a9a9a5601ac2f1732c9, State: Initialized, Role: FOLLOWER
I20260806 03:50:11.830996 2682 consensus_queue.cc:260] T 4c8bb61adcde46d3860a2e83cc3a0470 P 3f2f80aff6824a9a9a5601ac2f1732c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f2f80aff6824a9a9a5601ac2f1732c9" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 42305 } } peers { permanent_uuid: "879486408a1b4379b9154994c41e45e9" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 34471 } } peers { permanent_uuid: "fa87c812931045b48ed020a086658673" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 34953 } }
I20260806 03:50:11.831290 2682 ts_tablet_manager.cc:1428] T 4c8bb61adcde46d3860a2e83cc3a0470 P 3f2f80aff6824a9a9a5601ac2f1732c9: Time spent starting tablet: real 0.002s user 0.002s sys 0.004s
I20260806 03:50:11.834542 2829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:50:11.822614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.253.193:41771 (local address 127.11.253.254:42435)
0806 03:50:11.822851 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0806 03:50:11.822861 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 03:50:11.822939 (+ 78us) server_negotiation.cc:407] Connection header received
0806 03:50:11.823729 (+ 790us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:50:11.823737 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:50:11.823798 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:50:11.823972 (+ 174us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:50:11.824589 (+ 617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:50:11.825856 (+ 1267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:50:11.826957 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:50:11.827544 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:50:11.827664 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:50:11.830659 (+ 2995us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:50:11.830664 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:50:11.830665 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:50:11.830678 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:50:11.832691 (+ 2013us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:50:11.833124 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:50:11.833127 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:50:11.833128 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:50:11.833236 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:50:11.833496 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:50:11.833497 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:50:11.833497 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:50:11.833745 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:50:11.833839 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0806 03:50:11.834194 (+ 355us) server_negotiation.cc:299] Negotiation successful
0806 03:50:11.834332 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":186,"thread_start_us":100,"threads_started":1}
I20260806 03:50:11.834539 2828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:50:11.822495 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:42435 (local address 127.11.253.193:41771)
0806 03:50:11.822785 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0806 03:50:11.822829 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0806 03:50:11.823461 (+ 632us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:50:11.824008 (+ 547us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:50:11.824011 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:50:11.824273 (+ 262us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:50:11.824467 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:50:11.824471 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:50:11.826028 (+ 1557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:50:11.826029 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:50:11.826865 (+ 836us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:50:11.826867 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:50:11.827679 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:50:11.827681 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:50:11.827845 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:50:11.828548 (+ 703us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:50:11.828559 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:50:11.830494 (+ 1935us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:50:11.832794 (+ 2300us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:50:11.832801 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:50:11.832803 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:50:11.833037 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:50:11.833312 (+ 275us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:50:11.833313 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:50:11.833314 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:50:11.833424 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:50:11.833817 (+ 393us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:50:11.833824 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:50:11.834028 (+ 204us) client_negotiation.cc:769] Sending connection context
0806 03:50:11.834138 (+ 110us) client_negotiation.cc:240] Negotiation successful
0806 03:50:11.834332 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":243,"thread_start_us":104,"threads_started":1}
I20260806 03:50:11.835397 2824 heartbeater.cc:347] Connected to a master server at 127.11.253.254:42435
I20260806 03:50:11.835491 2824 heartbeater.cc:464] Registering TS with master...
I20260806 03:50:11.835683 2824 heartbeater.cc:511] Master 127.11.253.254:42435 requested a full tablet report, sending...
I20260806 03:50:11.836495 2569 ts_manager.cc:194] Registered new tserver with Master: 3f2f80aff6824a9a9a5601ac2f1732c9 (127.11.253.193:42305)
I20260806 03:50:11.837857 2569 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.253.193@KRBTEST.COM'} at 127.11.253.193:41771
I20260806 03:50:11.838949 2824 heartbeater.cc:503] Master 127.11.253.254:42435 was elected leader, sending a full tablet report...
Aug 06 03:50:11 dist-test-slave-6ntk krb5kdc[1775](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988211, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.253.254@KRBTEST.COM for kudu/127.11.253.193@KRBTEST.COM
I20260806 03:50:11.845693 2835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:50:11.837236 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55826 (local address 127.11.253.193:42305)
0806 03:50:11.837398 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0806 03:50:11.837402 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 03:50:11.837576 (+ 174us) server_negotiation.cc:407] Connection header received
0806 03:50:11.837675 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:50:11.837678 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:50:11.837697 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:50:11.837772 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:50:11.838327 (+ 555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:50:11.839391 (+ 1064us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:50:11.840223 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:50:11.840534 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:50:11.840613 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:50:11.842795 (+ 2182us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:50:11.842803 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:50:11.842804 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:50:11.842810 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:50:11.844332 (+ 1522us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:50:11.844789 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:50:11.844792 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:50:11.844792 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:50:11.844851 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:50:11.845086 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:50:11.845087 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:50:11.845088 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:50:11.845277 (+ 189us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:50:11.845366 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0806 03:50:11.845539 (+ 173us) server_negotiation.cc:299] Negotiation successful
0806 03:50:11.845622 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":75,"threads_started":1}
I20260806 03:50:11.845827 2834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:50:11.837186 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.193:42305 (local address 127.0.0.1:55826)
0806 03:50:11.837474 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0806 03:50:11.837504 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0806 03:50:11.837594 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:50:11.837775 (+ 181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:50:11.837781 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:50:11.838169 (+ 388us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:50:11.838252 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:50:11.838255 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:50:11.839497 (+ 1242us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:50:11.839497 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:50:11.840126 (+ 629us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:50:11.840128 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:50:11.840613 (+ 485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:50:11.840613 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:50:11.840696 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:50:11.841205 (+ 509us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:50:11.841211 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:50:11.842697 (+ 1486us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:50:11.844413 (+ 1716us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:50:11.844420 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:50:11.844422 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:50:11.844701 (+ 279us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:50:11.844916 (+ 215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:50:11.844917 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:50:11.844917 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:50:11.844988 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:50:11.845348 (+ 360us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:50:11.845355 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:50:11.845412 (+ 57us) client_negotiation.cc:769] Sending connection context
0806 03:50:11.845515 (+ 103us) client_negotiation.cc:240] Negotiation successful
0806 03:50:11.845658 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":263,"thread_start_us":107,"threads_started":1}
I20260806 03:50:11.846395 2734 tablet_service.cc:1514] Processing DeleteTablet for tablet 4c8bb61adcde46d3860a2e83cc3a0470 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 03:49:51 UTC) from {username='kudu', principal='kudu/127.11.253.254@KRBTEST.COM'} at 127.0.0.1:55826
I20260806 03:50:11.846956 2836 tablet_replica.cc:330] stopping tablet replica
I20260806 03:50:11.847071 2836 raft_consensus.cc:2229] T 4c8bb61adcde46d3860a2e83cc3a0470 P 3f2f80aff6824a9a9a5601ac2f1732c9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 03:50:11.847147 2836 raft_consensus.cc:2258] T 4c8bb61adcde46d3860a2e83cc3a0470 P 3f2f80aff6824a9a9a5601ac2f1732c9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 03:50:11.847472 2836 ts_tablet_manager.cc:1905] T 4c8bb61adcde46d3860a2e83cc3a0470 P 3f2f80aff6824a9a9a5601ac2f1732c9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 03:50:11.849550 2836 ts_tablet_manager.cc:1918] T 4c8bb61adcde46d3860a2e83cc3a0470 P 3f2f80aff6824a9a9a5601ac2f1732c9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 03:50:11.849614 2836 log.cc:1196] T 4c8bb61adcde46d3860a2e83cc3a0470 P 3f2f80aff6824a9a9a5601ac2f1732c9: Deleting WAL directory at /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/minicluster-data/ts-0/wal/wals/4c8bb61adcde46d3860a2e83cc3a0470
I20260806 03:50:11.849900 2836 ts_tablet_manager.cc:1939] T 4c8bb61adcde46d3860a2e83cc3a0470 P 3f2f80aff6824a9a9a5601ac2f1732c9: Deleting consensus metadata
I20260806 03:50:11.850455 2550 catalog_manager.cc:4841] TS 3f2f80aff6824a9a9a5601ac2f1732c9 (127.11.253.193:42305): tablet 4c8bb61adcde46d3860a2e83cc3a0470 (table test-table [id=c9cb2160fb03459eb091c67035a781df]) successfully deleted
W20260806 03:50:11.915781 2831 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:50:11.916039 2831 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:50:11.916078 2831 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:50:11.917243 2831 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:50:11.917318 2831 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:50:11.917363 2831 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.253.194
I20260806 03:50:11.918862 2831 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.253.212:40037
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.194.keytab
--principal=kudu/127.11.253.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.253.194:34953
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.253.194
--webserver_port=39327
--tserver_master_addrs=127.11.253.254:42435
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2831
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.253.194
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:50:11.919137 2831 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:50:11.919377 2831 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:50:11.924535 2843 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:50:11.924638 2845 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:50:11.924746 2831 server_base.cc:1034] running on GCE node
W20260806 03:50:11.924557 2842 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:50:11.925016 2831 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:50:11.925221 2831 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:50:11.926362 2831 hybrid_clock.cc:648] HybridClock initialized: now 1785988211926343 us; error 43 us; skew 500 ppm
Aug 06 03:50:11 dist-test-slave-6ntk krb5kdc[1775](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988211, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.253.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:50:11.928890 2831 init.cc:377] Logged in from keytab as kudu/127.11.253.194@KRBTEST.COM (short username kudu)
I20260806 03:50:11.929291 2831 webserver.cc:460] Webserver started at http://127.11.253.194:39327/ using document root <none> and password file <none>
I20260806 03:50:11.929484 2831 fs_manager.cc:359] Metadata directory not provided
I20260806 03:50:11.929538 2831 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:50:11.930975 2831 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 03:50:11.931610 2855 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:50:11.931806 2831 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 03:50:11.931896 2831 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/minicluster-data/ts-1/data,/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/minicluster-data/ts-1/wal
uuid: "fa87c812931045b48ed020a086658673"
format_stamp: "Formatted at 2026-08-06 03:49:51 on dist-test-slave-6ntk"
server_key: "090c6966b5b8a8ba039f3e35b75630e7"
server_key_iv: "0498c893b2e7d2f6c591c3e1ceb79195"
server_key_version: "encryptionkey@0"
I20260806 03:50:11.932246 2831 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:50:11.941228 2831 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:50:11.943670 2831 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:50:11.943796 2831 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:50:11.944003 2831 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 03:50:11.944386 2831 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 03:50:11.944424 2831 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 03:50:11.944478 2831 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 03:50:11.944507 2831 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 03:50:11.950613 2831 rpc_server.cc:307] RPC server started. Bound to: 127.11.253.194:34953
I20260806 03:50:11.950701 3012 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.253.194:34953 every 8 connection(s)
I20260806 03:50:11.951047 2831 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/minicluster-data/ts-1/data/info.pb
I20260806 03:50:11.954478 12279 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskToVWRe/build/release/bin/kudu as pid 2831
I20260806 03:50:11.954811 12279 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskToVWRe/build/release/bin/kudu
/tmp/dist-test-taskToVWRe/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.253.195:34471
--local_ip_for_outbound_sockets=127.11.253.195
--tserver_master_addrs=127.11.253.254:42435
--webserver_port=43257
--webserver_interface=127.11.253.195
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.195.keytab
--principal=kudu/127.11.253.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.253.212:40037
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 03:50:11 dist-test-slave-6ntk krb5kdc[1775](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988211, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.253.194@KRBTEST.COM for kudu/127.11.253.254@KRBTEST.COM
I20260806 03:50:11.961597 2829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:50:11.951886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.253.194:55763 (local address 127.11.253.254:42435)
0806 03:50:11.951952 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0806 03:50:11.951954 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:50:11.952147 (+ 193us) server_negotiation.cc:407] Connection header received
0806 03:50:11.952677 (+ 530us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:50:11.952678 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:50:11.952693 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:50:11.952772 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:50:11.953756 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:50:11.954320 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:50:11.955821 (+ 1501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:50:11.956032 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:50:11.956098 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:50:11.958586 (+ 2488us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:50:11.958587 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:50:11.958588 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:50:11.958595 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:50:11.960188 (+ 1593us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:50:11.960613 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:50:11.960614 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:50:11.960615 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:50:11.960675 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:50:11.960934 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:50:11.960935 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:50:11.960935 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:50:11.961051 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:50:11.961118 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0806 03:50:11.961407 (+ 289us) server_negotiation.cc:299] Negotiation successful
0806 03:50:11.961505 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260806 03:50:11.961798 3017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:50:11.951882 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:42435 (local address 127.11.253.194:55763)
0806 03:50:11.952032 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0806 03:50:11.952069 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0806 03:50:11.952564 (+ 495us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:50:11.952851 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:50:11.952855 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:50:11.953463 (+ 608us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:50:11.953658 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:50:11.953662 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:50:11.954406 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:50:11.954407 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:50:11.955719 (+ 1312us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:50:11.955722 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:50:11.956120 (+ 398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:50:11.956122 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:50:11.956280 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:50:11.956939 (+ 659us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:50:11.956948 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:50:11.958496 (+ 1548us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:50:11.960293 (+ 1797us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:50:11.960301 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:50:11.960304 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:50:11.960537 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:50:11.960741 (+ 204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:50:11.960741 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:50:11.960742 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:50:11.960861 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:50:11.961119 (+ 258us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:50:11.961128 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:50:11.961270 (+ 142us) client_negotiation.cc:769] Sending connection context
0806 03:50:11.961373 (+ 103us) client_negotiation.cc:240] Negotiation successful
0806 03:50:11.961535 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":85,"threads_started":1}
I20260806 03:50:11.962306 3013 heartbeater.cc:347] Connected to a master server at 127.11.253.254:42435
I20260806 03:50:11.962400 3013 heartbeater.cc:464] Registering TS with master...
I20260806 03:50:11.962597 3013 heartbeater.cc:511] Master 127.11.253.254:42435 requested a full tablet report, sending...
I20260806 03:50:11.963007 2569 ts_manager.cc:194] Registered new tserver with Master: fa87c812931045b48ed020a086658673 (127.11.253.194:34953)
I20260806 03:50:11.963390 2569 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.253.194@KRBTEST.COM'} at 127.11.253.194:55763
W20260806 03:50:12.041917 3018 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:50:12.042174 3018 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:50:12.042214 3018 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:50:12.043404 3018 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:50:12.043473 3018 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:50:12.043519 3018 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.253.195
I20260806 03:50:12.045091 3018 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.253.212:40037
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.195.keytab
--principal=kudu/127.11.253.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.253.195:34471
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.253.195
--webserver_port=43257
--tserver_master_addrs=127.11.253.254:42435
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3018
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.253.195
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:50:12.045401 3018 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:50:12.045652 3018 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 03:50:12.051030 3018 server_base.cc:1034] running on GCE node
W20260806 03:50:12.051194 3024 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:50:12.051088 3025 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:50:12.051100 3028 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:50:12.051607 3018 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:50:12.051865 3018 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:50:12.053006 3018 hybrid_clock.cc:648] HybridClock initialized: now 1785988212052996 us; error 35 us; skew 500 ppm
Aug 06 03:50:12 dist-test-slave-6ntk krb5kdc[1775](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988212, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.253.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:50:12.055667 3018 init.cc:377] Logged in from keytab as kudu/127.11.253.195@KRBTEST.COM (short username kudu)
I20260806 03:50:12.056102 3018 webserver.cc:460] Webserver started at http://127.11.253.195:43257/ using document root <none> and password file <none>
I20260806 03:50:12.056347 3018 fs_manager.cc:359] Metadata directory not provided
I20260806 03:50:12.056452 3018 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:50:12.057898 3018 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 03:50:12.058604 3035 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:50:12.058874 3018 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 03:50:12.058982 3018 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/minicluster-data/ts-2/data,/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/minicluster-data/ts-2/wal
uuid: "879486408a1b4379b9154994c41e45e9"
format_stamp: "Formatted at 2026-08-06 03:49:51 on dist-test-slave-6ntk"
server_key: "29100263602bca389eb0279894831131"
server_key_iv: "e8d05ae3aaccbc0c2abdd6af0c30829a"
server_key_version: "encryptionkey@0"
I20260806 03:50:12.059387 3018 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:50:12.080928 3018 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:50:12.083256 3018 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:50:12.083432 3018 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:50:12.083683 3018 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 03:50:12.084009 3018 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 03:50:12.084060 3018 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 03:50:12.084105 3018 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 03:50:12.084131 3018 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 03:50:12.090273 3018 rpc_server.cc:307] RPC server started. Bound to: 127.11.253.195:34471
I20260806 03:50:12.090340 3187 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.253.195:34471 every 8 connection(s)
I20260806 03:50:12.090727 3018 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/minicluster-data/ts-2/data/info.pb
I20260806 03:50:12.091877 12279 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskToVWRe/build/release/bin/kudu as pid 3018
Aug 06 03:50:12 dist-test-slave-6ntk krb5kdc[1775](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988212, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.253.195@KRBTEST.COM for kudu/127.11.253.254@KRBTEST.COM
I20260806 03:50:12.100078 3192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:50:12.092313 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38444 (local address 127.11.253.254:42435)
0806 03:50:12.092729 (+ 416us) server_negotiation.cc:206] Beginning negotiation
0806 03:50:12.092730 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 03:50:12.092736 (+ 6us) server_negotiation.cc:407] Connection header received
0806 03:50:12.092756 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:50:12.092756 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:50:12.092766 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:50:12.092835 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:50:12.093602 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:50:12.094192 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:50:12.095041 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:50:12.095251 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:50:12.095323 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:50:12.096934 (+ 1611us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:50:12.096935 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:50:12.096935 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:50:12.096941 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:50:12.098887 (+ 1946us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:50:12.099267 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:50:12.099268 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:50:12.099269 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:50:12.099336 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:50:12.099539 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:50:12.099540 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:50:12.099541 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:50:12.099708 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:50:12.099900 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0806 03:50:12.099942 (+ 42us) server_negotiation.cc:299] Negotiation successful
0806 03:50:12.100007 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":79,"threads_started":1}
I20260806 03:50:12.101293 2829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:50:12.091479 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.253.195:46639 (local address 127.11.253.254:42435)
0806 03:50:12.091564 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0806 03:50:12.091566 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:50:12.091852 (+ 286us) server_negotiation.cc:407] Connection header received
0806 03:50:12.092685 (+ 833us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:50:12.092686 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:50:12.092699 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:50:12.092783 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:50:12.093851 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:50:12.094349 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:50:12.095395 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:50:12.095638 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:50:12.095704 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:50:12.098086 (+ 2382us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:50:12.098087 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:50:12.098087 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:50:12.098094 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:50:12.099780 (+ 1686us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:50:12.100308 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:50:12.100308 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:50:12.100309 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:50:12.100364 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:50:12.100643 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:50:12.100644 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:50:12.100644 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:50:12.100754 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:50:12.100827 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0806 03:50:12.101121 (+ 294us) server_negotiation.cc:299] Negotiation successful
0806 03:50:12.101204 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260806 03:50:12.101495 3191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:50:12.091484 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:42435 (local address 127.11.253.195:46639)
0806 03:50:12.091695 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0806 03:50:12.091750 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0806 03:50:12.092577 (+ 827us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:50:12.092833 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:50:12.092837 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:50:12.093509 (+ 672us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:50:12.093765 (+ 256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:50:12.093769 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:50:12.094445 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:50:12.094446 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:50:12.095312 (+ 866us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:50:12.095314 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:50:12.095722 (+ 408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:50:12.095723 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:50:12.095910 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:50:12.096415 (+ 505us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:50:12.096425 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:50:12.098014 (+ 1589us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:50:12.099866 (+ 1852us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:50:12.099874 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:50:12.099876 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:50:12.100201 (+ 325us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:50:12.100433 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:50:12.100433 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:50:12.100434 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:50:12.100568 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:50:12.100833 (+ 265us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:50:12.100841 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:50:12.100987 (+ 146us) client_negotiation.cc:769] Sending connection context
0806 03:50:12.101100 (+ 113us) client_negotiation.cc:240] Negotiation successful
0806 03:50:12.101266 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":101,"threads_started":1}
I20260806 03:50:12.101971 3189 heartbeater.cc:347] Connected to a master server at 127.11.253.254:42435
I20260806 03:50:12.102088 3189 heartbeater.cc:464] Registering TS with master...
I20260806 03:50:12.102275 3189 heartbeater.cc:511] Master 127.11.253.254:42435 requested a full tablet report, sending...
I20260806 03:50:12.102643 2569 ts_manager.cc:194] Registered new tserver with Master: 879486408a1b4379b9154994c41e45e9 (127.11.253.195:34471)
I20260806 03:50:12.103012 2569 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.253.195@KRBTEST.COM'} at 127.11.253.195:46639
I20260806 03:50:12.111917 12279 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 03:50:12.114686 2829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:50:12.112333 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38456 (local address 127.11.253.254:42435)
0806 03:50:12.112397 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0806 03:50:12.112399 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:50:12.112530 (+ 131us) server_negotiation.cc:407] Connection header received
0806 03:50:12.112613 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:50:12.112614 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:50:12.112652 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:50:12.112706 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 03:50:12.112810 (+ 104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:50:12.113269 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:50:12.114126 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:50:12.114300 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:50:12.114352 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:50:12.114473 (+ 121us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 03:50:12.114517 (+ 44us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0806 03:50:12.114636 (+ 119us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:38456: token expired
Metrics: {"server-negotiator.queue_time_us":39}
W20260806 03:50:12.114753 2829 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:38456: token expired
W20260806 03:50:12.115024 2464 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.11.253.254:42435)
I20260806 03:50:12.121994 2829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:50:12.115305 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38468 (local address 127.11.253.254:42435)
0806 03:50:12.115356 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0806 03:50:12.115358 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:50:12.115370 (+ 12us) server_negotiation.cc:407] Connection header received
0806 03:50:12.115437 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:50:12.115438 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:50:12.115450 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:50:12.115518 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:50:12.116102 (+ 584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:50:12.116608 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:50:12.117436 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:50:12.117643 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:50:12.117746 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:50:12.119112 (+ 1366us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:50:12.119114 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:50:12.119114 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:50:12.119123 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:50:12.120586 (+ 1463us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:50:12.121047 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:50:12.121048 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:50:12.121049 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:50:12.121126 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:50:12.121370 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:50:12.121371 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:50:12.121371 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:50:12.121503 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:50:12.121635 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0806 03:50:12.121772 (+ 137us) server_negotiation.cc:299] Negotiation successful
0806 03:50:12.121855 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260806 03:50:12.123253 2569 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38468:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 03:50:12.125017 2569 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 03:50:12.127372 2734 tablet_service.cc:1467] Processing CreateTablet for tablet 25c0a2833d9e40839c03d7070c39fef6 (DEFAULT_TABLE table=test-table [id=feaa80deed054645bb08caffc5a3e567]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 03:50:12.127595 2734 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 25c0a2833d9e40839c03d7070c39fef6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:50:12.129034 2682 tablet_bootstrap.cc:492] T 25c0a2833d9e40839c03d7070c39fef6 P 3f2f80aff6824a9a9a5601ac2f1732c9: Bootstrap starting.
I20260806 03:50:12.129434 2682 tablet_bootstrap.cc:654] T 25c0a2833d9e40839c03d7070c39fef6 P 3f2f80aff6824a9a9a5601ac2f1732c9: Neither blocks nor log segments found. Creating new log.
I20260806 03:50:12.130146 2682 tablet_bootstrap.cc:492] T 25c0a2833d9e40839c03d7070c39fef6 P 3f2f80aff6824a9a9a5601ac2f1732c9: No bootstrap required, opened a new log
I20260806 03:50:12.130210 2682 ts_tablet_manager.cc:1397] T 25c0a2833d9e40839c03d7070c39fef6 P 3f2f80aff6824a9a9a5601ac2f1732c9: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260806 03:50:12.130648 2682 raft_consensus.cc:348] T 25c0a2833d9e40839c03d7070c39fef6 P 3f2f80aff6824a9a9a5601ac2f1732c9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa87c812931045b48ed020a086658673" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 34953 } } peers { permanent_uuid: "879486408a1b4379b9154994c41e45e9" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 34471 } } peers { permanent_uuid: "3f2f80aff6824a9a9a5601ac2f1732c9" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 42305 } }
I20260806 03:50:12.130750 2682 raft_consensus.cc:374] T 25c0a2833d9e40839c03d7070c39fef6 P 3f2f80aff6824a9a9a5601ac2f1732c9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:50:12.130808 2682 raft_consensus.cc:729] T 25c0a2833d9e40839c03d7070c39fef6 P 3f2f80aff6824a9a9a5601ac2f1732c9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f2f80aff6824a9a9a5601ac2f1732c9, State: Initialized, Role: FOLLOWER
I20260806 03:50:12.130875 2682 consensus_queue.cc:260] T 25c0a2833d9e40839c03d7070c39fef6 P 3f2f80aff6824a9a9a5601ac2f1732c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa87c812931045b48ed020a086658673" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 34953 } } peers { permanent_uuid: "879486408a1b4379b9154994c41e45e9" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 34471 } } peers { permanent_uuid: "3f2f80aff6824a9a9a5601ac2f1732c9" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 42305 } }
I20260806 03:50:12.130988 2682 ts_tablet_manager.cc:1428] T 25c0a2833d9e40839c03d7070c39fef6 P 3f2f80aff6824a9a9a5601ac2f1732c9: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260806 03:50:12.131922 3196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:50:12.127138 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.195:34471 (local address 127.0.0.1:45356)
0806 03:50:12.127640 (+ 502us) negotiation.cc:107] Waiting for socket to connect
0806 03:50:12.127668 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0806 03:50:12.127725 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:50:12.128049 (+ 324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:50:12.128050 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:50:12.128053 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 03:50:12.128107 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:50:12.128109 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:50:12.129255 (+ 1146us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:50:12.129256 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:50:12.130649 (+ 1393us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:50:12.130652 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:50:12.131598 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:50:12.131599 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:50:12.131670 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:50:12.131706 (+ 36us) client_negotiation.cc:769] Sending connection context
0806 03:50:12.131770 (+ 64us) client_negotiation.cc:240] Negotiation successful
0806 03:50:12.131825 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":477,"mutex_wait_us":272,"thread_start_us":130,"threads_started":1}
I20260806 03:50:12.131937 3197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:50:12.127580 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45356 (local address 127.11.253.195:34471)
0806 03:50:12.127844 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0806 03:50:12.127849 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 03:50:12.127859 (+ 10us) server_negotiation.cc:407] Connection header received
0806 03:50:12.127897 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:50:12.127898 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:50:12.127913 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:50:12.128011 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 03:50:12.128226 (+ 215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:50:12.129137 (+ 911us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:50:12.130745 (+ 1608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:50:12.131506 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:50:12.131716 (+ 210us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:50:12.131794 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0806 03:50:12.131823 (+ 29us) server_negotiation.cc:299] Negotiation successful
0806 03:50:12.131867 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":218,"thread_start_us":97,"threads_started":1}
I20260806 03:50:12.132728 3100 tablet_service.cc:1467] Processing CreateTablet for tablet 25c0a2833d9e40839c03d7070c39fef6 (DEFAULT_TABLE table=test-table [id=feaa80deed054645bb08caffc5a3e567]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 03:50:12.133056 2834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:50:12.127138 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.194:34953 (local address 127.0.0.1:57534)
0806 03:50:12.127314 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0806 03:50:12.127325 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 03:50:12.127367 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:50:12.129281 (+ 1914us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:50:12.129282 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:50:12.129284 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 03:50:12.129318 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:50:12.129319 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:50:12.130616 (+ 1297us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:50:12.130617 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:50:12.131700 (+ 1083us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:50:12.131703 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:50:12.132768 (+ 1065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:50:12.132769 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:50:12.132852 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:50:12.132857 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 03:50:12.132930 (+ 73us) client_negotiation.cc:240] Negotiation successful
0806 03:50:12.132981 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151}
I20260806 03:50:12.133086 3195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:50:12.127339 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57534 (local address 127.11.253.194:34953)
0806 03:50:12.129104 (+ 1765us) server_negotiation.cc:206] Beginning negotiation
0806 03:50:12.129108 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 03:50:12.129119 (+ 11us) server_negotiation.cc:407] Connection header received
0806 03:50:12.129158 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:50:12.129160 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:50:12.129177 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:50:12.129401 (+ 224us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 03:50:12.129491 (+ 90us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:50:12.130530 (+ 1039us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:50:12.131824 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:50:12.132658 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:50:12.132768 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:50:12.132828 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0806 03:50:12.132947 (+ 119us) server_negotiation.cc:299] Negotiation successful
0806 03:50:12.132992 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1724,"thread_start_us":132,"threads_started":1}
I20260806 03:50:12.133100 3100 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 25c0a2833d9e40839c03d7070c39fef6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:50:12.133767 2917 tablet_service.cc:1467] Processing CreateTablet for tablet 25c0a2833d9e40839c03d7070c39fef6 (DEFAULT_TABLE table=test-table [id=feaa80deed054645bb08caffc5a3e567]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 03:50:12.134158 2917 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 25c0a2833d9e40839c03d7070c39fef6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:50:12.136155 3199 tablet_bootstrap.cc:492] T 25c0a2833d9e40839c03d7070c39fef6 P 879486408a1b4379b9154994c41e45e9: Bootstrap starting.
I20260806 03:50:12.137017 3199 tablet_bootstrap.cc:654] T 25c0a2833d9e40839c03d7070c39fef6 P 879486408a1b4379b9154994c41e45e9: Neither blocks nor log segments found. Creating new log.
I20260806 03:50:12.137748 3199 log.cc:824] T 25c0a2833d9e40839c03d7070c39fef6 P 879486408a1b4379b9154994c41e45e9: Log is configured to *not* fsync() on all Append() calls
I20260806 03:50:12.138026 3201 tablet_bootstrap.cc:492] T 25c0a2833d9e40839c03d7070c39fef6 P fa87c812931045b48ed020a086658673: Bootstrap starting.
I20260806 03:50:12.138656 3201 tablet_bootstrap.cc:654] T 25c0a2833d9e40839c03d7070c39fef6 P fa87c812931045b48ed020a086658673: Neither blocks nor log segments found. Creating new log.
I20260806 03:50:12.138738 3199 tablet_bootstrap.cc:492] T 25c0a2833d9e40839c03d7070c39fef6 P 879486408a1b4379b9154994c41e45e9: No bootstrap required, opened a new log
I20260806 03:50:12.138806 3199 ts_tablet_manager.cc:1397] T 25c0a2833d9e40839c03d7070c39fef6 P 879486408a1b4379b9154994c41e45e9: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 03:50:12.139139 3201 log.cc:824] T 25c0a2833d9e40839c03d7070c39fef6 P fa87c812931045b48ed020a086658673: Log is configured to *not* fsync() on all Append() calls
I20260806 03:50:12.139923 3201 tablet_bootstrap.cc:492] T 25c0a2833d9e40839c03d7070c39fef6 P fa87c812931045b48ed020a086658673: No bootstrap required, opened a new log
I20260806 03:50:12.139976 3201 ts_tablet_manager.cc:1397] T 25c0a2833d9e40839c03d7070c39fef6 P fa87c812931045b48ed020a086658673: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 03:50:12.140492 3199 raft_consensus.cc:348] T 25c0a2833d9e40839c03d7070c39fef6 P 879486408a1b4379b9154994c41e45e9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa87c812931045b48ed020a086658673" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 34953 } } peers { permanent_uuid: "879486408a1b4379b9154994c41e45e9" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 34471 } } peers { permanent_uuid: "3f2f80aff6824a9a9a5601ac2f1732c9" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 42305 } }
I20260806 03:50:12.140648 3199 raft_consensus.cc:374] T 25c0a2833d9e40839c03d7070c39fef6 P 879486408a1b4379b9154994c41e45e9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:50:12.140707 3199 raft_consensus.cc:729] T 25c0a2833d9e40839c03d7070c39fef6 P 879486408a1b4379b9154994c41e45e9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 879486408a1b4379b9154994c41e45e9, State: Initialized, Role: FOLLOWER
I20260806 03:50:12.140831 3199 consensus_queue.cc:260] T 25c0a2833d9e40839c03d7070c39fef6 P 879486408a1b4379b9154994c41e45e9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa87c812931045b48ed020a086658673" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 34953 } } peers { permanent_uuid: "879486408a1b4379b9154994c41e45e9" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 34471 } } peers { permanent_uuid: "3f2f80aff6824a9a9a5601ac2f1732c9" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 42305 } }
I20260806 03:50:12.141126 3199 ts_tablet_manager.cc:1428] T 25c0a2833d9e40839c03d7070c39fef6 P 879486408a1b4379b9154994c41e45e9: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260806 03:50:12.141155 3189 heartbeater.cc:503] Master 127.11.253.254:42435 was elected leader, sending a full tablet report...
I20260806 03:50:12.141654 3201 raft_consensus.cc:348] T 25c0a2833d9e40839c03d7070c39fef6 P fa87c812931045b48ed020a086658673 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa87c812931045b48ed020a086658673" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 34953 } } peers { permanent_uuid: "879486408a1b4379b9154994c41e45e9" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 34471 } } peers { permanent_uuid: "3f2f80aff6824a9a9a5601ac2f1732c9" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 42305 } }
I20260806 03:50:12.141858 3201 raft_consensus.cc:374] T 25c0a2833d9e40839c03d7070c39fef6 P fa87c812931045b48ed020a086658673 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:50:12.141939 3201 raft_consensus.cc:729] T 25c0a2833d9e40839c03d7070c39fef6 P fa87c812931045b48ed020a086658673 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa87c812931045b48ed020a086658673, State: Initialized, Role: FOLLOWER
I20260806 03:50:12.142061 3201 consensus_queue.cc:260] T 25c0a2833d9e40839c03d7070c39fef6 P fa87c812931045b48ed020a086658673 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa87c812931045b48ed020a086658673" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 34953 } } peers { permanent_uuid: "879486408a1b4379b9154994c41e45e9" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 34471 } } peers { permanent_uuid: "3f2f80aff6824a9a9a5601ac2f1732c9" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 42305 } }
I20260806 03:50:12.142331 3201 ts_tablet_manager.cc:1428] T 25c0a2833d9e40839c03d7070c39fef6 P fa87c812931045b48ed020a086658673: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260806 03:50:12.142364 3013 heartbeater.cc:503] Master 127.11.253.254:42435 was elected leader, sending a full tablet report...
W20260806 03:50:12.201642 3016 tablet.cc:2369] T 25c0a2833d9e40839c03d7070c39fef6 P fa87c812931045b48ed020a086658673: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 03:50:12.322602 2825 tablet.cc:2369] T 25c0a2833d9e40839c03d7070c39fef6 P 3f2f80aff6824a9a9a5601ac2f1732c9: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 03:50:12.341328 3190 tablet.cc:2369] T 25c0a2833d9e40839c03d7070c39fef6 P 879486408a1b4379b9154994c41e45e9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 03:50:12.479943 2832 raft_consensus.cc:482] T 25c0a2833d9e40839c03d7070c39fef6 P 3f2f80aff6824a9a9a5601ac2f1732c9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 03:50:12.480109 2832 raft_consensus.cc:504] T 25c0a2833d9e40839c03d7070c39fef6 P 3f2f80aff6824a9a9a5601ac2f1732c9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa87c812931045b48ed020a086658673" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 34953 } } peers { permanent_uuid: "879486408a1b4379b9154994c41e45e9" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 34471 } } peers { permanent_uuid: "3f2f80aff6824a9a9a5601ac2f1732c9" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 42305 } }
I20260806 03:50:12.480676 2832 leader_election.cc:288] T 25c0a2833d9e40839c03d7070c39fef6 P 3f2f80aff6824a9a9a5601ac2f1732c9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fa87c812931045b48ed020a086658673 (127.11.253.194:34953), 879486408a1b4379b9154994c41e45e9 (127.11.253.195:34471)
I20260806 03:50:12.484752 3195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:50:12.480938 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.253.193:46679 (local address 127.11.253.194:34953)
0806 03:50:12.480989 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0806 03:50:12.480991 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:50:12.481190 (+ 199us) server_negotiation.cc:407] Connection header received
0806 03:50:12.481288 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:50:12.481289 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:50:12.481303 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:50:12.481362 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 03:50:12.481534 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:50:12.482314 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:50:12.483811 (+ 1497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:50:12.484362 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:50:12.484440 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:50:12.484449 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0806 03:50:12.484641 (+ 192us) server_negotiation.cc:299] Negotiation successful
0806 03:50:12.484679 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260806 03:50:12.484752 3197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:50:12.481101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.253.193:54029 (local address 127.11.253.195:34471)
0806 03:50:12.481148 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0806 03:50:12.481150 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:50:12.481159 (+ 9us) server_negotiation.cc:407] Connection header received
0806 03:50:12.481288 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:50:12.481289 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:50:12.481300 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:50:12.481362 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 03:50:12.481534 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:50:12.482296 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:50:12.483800 (+ 1504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:50:12.484366 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:50:12.484440 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:50:12.484449 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0806 03:50:12.484640 (+ 191us) server_negotiation.cc:299] Negotiation successful
0806 03:50:12.484679 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 03:50:12.484792 3208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:50:12.480850 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.195:34471 (local address 127.11.253.193:54029)
0806 03:50:12.481059 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0806 03:50:12.481071 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 03:50:12.481147 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:50:12.481376 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:50:12.481377 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:50:12.481378 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 03:50:12.481460 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:50:12.481461 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:50:12.482392 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:50:12.482393 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:50:12.483706 (+ 1313us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:50:12.483709 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:50:12.484460 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:50:12.484460 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:50:12.484535 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:50:12.484564 (+ 29us) client_negotiation.cc:769] Sending connection context
0806 03:50:12.484625 (+ 61us) client_negotiation.cc:240] Negotiation successful
0806 03:50:12.484687 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":193,"mutex_wait_us":70,"thread_start_us":97,"threads_started":1}
I20260806 03:50:12.484980 3209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:50:12.480849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.194:34953 (local address 127.11.253.193:46679)
0806 03:50:12.481111 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0806 03:50:12.481120 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 03:50:12.481179 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:50:12.481374 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:50:12.481375 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:50:12.481377 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 03:50:12.481459 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:50:12.481461 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:50:12.482392 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:50:12.482393 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:50:12.483706 (+ 1313us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:50:12.483709 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:50:12.484456 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:50:12.484457 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:50:12.484535 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:50:12.484566 (+ 31us) client_negotiation.cc:769] Sending connection context
0806 03:50:12.484625 (+ 59us) client_negotiation.cc:240] Negotiation successful
0806 03:50:12.484687 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":248,"mutex_wait_us":108,"thread_start_us":87,"threads_started":1}
I20260806 03:50:12.485164 3127 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "25c0a2833d9e40839c03d7070c39fef6" candidate_uuid: "3f2f80aff6824a9a9a5601ac2f1732c9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "879486408a1b4379b9154994c41e45e9" is_pre_election: true
I20260806 03:50:12.485350 3127 raft_consensus.cc:2454] T 25c0a2833d9e40839c03d7070c39fef6 P 879486408a1b4379b9154994c41e45e9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f2f80aff6824a9a9a5601ac2f1732c9 in term 0.
I20260806 03:50:12.485404 2944 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "25c0a2833d9e40839c03d7070c39fef6" candidate_uuid: "3f2f80aff6824a9a9a5601ac2f1732c9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa87c812931045b48ed020a086658673" is_pre_election: true
I20260806 03:50:12.485565 2944 raft_consensus.cc:2454] T 25c0a2833d9e40839c03d7070c39fef6 P fa87c812931045b48ed020a086658673 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f2f80aff6824a9a9a5601ac2f1732c9 in term 0.
I20260806 03:50:12.485605 2675 leader_election.cc:302] T 25c0a2833d9e40839c03d7070c39fef6 P 3f2f80aff6824a9a9a5601ac2f1732c9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f2f80aff6824a9a9a5601ac2f1732c9, 879486408a1b4379b9154994c41e45e9; no voters:
I20260806 03:50:12.485745 2832 raft_consensus.cc:2781] T 25c0a2833d9e40839c03d7070c39fef6 P 3f2f80aff6824a9a9a5601ac2f1732c9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 03:50:12.485811 2832 raft_consensus.cc:482] T 25c0a2833d9e40839c03d7070c39fef6 P 3f2f80aff6824a9a9a5601ac2f1732c9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 03:50:12.485846 2832 raft_consensus.cc:3037] T 25c0a2833d9e40839c03d7070c39fef6 P 3f2f80aff6824a9a9a5601ac2f1732c9 [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:50:12.486603 2832 raft_consensus.cc:504] T 25c0a2833d9e40839c03d7070c39fef6 P 3f2f80aff6824a9a9a5601ac2f1732c9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa87c812931045b48ed020a086658673" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 34953 } } peers { permanent_uuid: "879486408a1b4379b9154994c41e45e9" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 34471 } } peers { permanent_uuid: "3f2f80aff6824a9a9a5601ac2f1732c9" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 42305 } }
I20260806 03:50:12.486763 2832 leader_election.cc:288] T 25c0a2833d9e40839c03d7070c39fef6 P 3f2f80aff6824a9a9a5601ac2f1732c9 [CANDIDATE]: Term 1 election: Requested vote from peers fa87c812931045b48ed020a086658673 (127.11.253.194:34953), 879486408a1b4379b9154994c41e45e9 (127.11.253.195:34471)
I20260806 03:50:12.486931 3127 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "25c0a2833d9e40839c03d7070c39fef6" candidate_uuid: "3f2f80aff6824a9a9a5601ac2f1732c9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "879486408a1b4379b9154994c41e45e9"
I20260806 03:50:12.487008 3127 raft_consensus.cc:3037] T 25c0a2833d9e40839c03d7070c39fef6 P 879486408a1b4379b9154994c41e45e9 [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:50:12.487702 3127 raft_consensus.cc:2454] T 25c0a2833d9e40839c03d7070c39fef6 P 879486408a1b4379b9154994c41e45e9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f2f80aff6824a9a9a5601ac2f1732c9 in term 1.
I20260806 03:50:12.487818 2944 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "25c0a2833d9e40839c03d7070c39fef6" candidate_uuid: "3f2f80aff6824a9a9a5601ac2f1732c9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa87c812931045b48ed020a086658673"
I20260806 03:50:12.487884 2675 leader_election.cc:302] T 25c0a2833d9e40839c03d7070c39fef6 P 3f2f80aff6824a9a9a5601ac2f1732c9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f2f80aff6824a9a9a5601ac2f1732c9, 879486408a1b4379b9154994c41e45e9; no voters:
I20260806 03:50:12.487928 2944 raft_consensus.cc:3037] T 25c0a2833d9e40839c03d7070c39fef6 P fa87c812931045b48ed020a086658673 [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:50:12.487980 2832 raft_consensus.cc:2781] T 25c0a2833d9e40839c03d7070c39fef6 P 3f2f80aff6824a9a9a5601ac2f1732c9 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 03:50:12.488111 2832 raft_consensus.cc:686] T 25c0a2833d9e40839c03d7070c39fef6 P 3f2f80aff6824a9a9a5601ac2f1732c9 [term 1 LEADER]: Becoming Leader. State: Replica: 3f2f80aff6824a9a9a5601ac2f1732c9, State: Running, Role: LEADER
I20260806 03:50:12.488188 2832 consensus_queue.cc:237] T 25c0a2833d9e40839c03d7070c39fef6 P 3f2f80aff6824a9a9a5601ac2f1732c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa87c812931045b48ed020a086658673" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 34953 } } peers { permanent_uuid: "879486408a1b4379b9154994c41e45e9" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 34471 } } peers { permanent_uuid: "3f2f80aff6824a9a9a5601ac2f1732c9" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 42305 } }
I20260806 03:50:12.488816 2944 raft_consensus.cc:2454] T 25c0a2833d9e40839c03d7070c39fef6 P fa87c812931045b48ed020a086658673 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f2f80aff6824a9a9a5601ac2f1732c9 in term 1.
I20260806 03:50:12.489398 2567 catalog_manager.cc:5495] T 25c0a2833d9e40839c03d7070c39fef6 P 3f2f80aff6824a9a9a5601ac2f1732c9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3f2f80aff6824a9a9a5601ac2f1732c9 (127.11.253.193). New cstate: current_term: 1 leader_uuid: "3f2f80aff6824a9a9a5601ac2f1732c9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa87c812931045b48ed020a086658673" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 34953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "879486408a1b4379b9154994c41e45e9" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 34471 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3f2f80aff6824a9a9a5601ac2f1732c9" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 42305 } health_report { overall_health: HEALTHY } } }
I20260806 03:50:12.572737 3212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:50:12.570384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55830 (local address 127.11.253.193:42305)
0806 03:50:12.570589 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0806 03:50:12.570591 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:50:12.570599 (+ 8us) server_negotiation.cc:407] Connection header received
0806 03:50:12.570641 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:50:12.570642 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:50:12.570662 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:50:12.570738 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 03:50:12.570861 (+ 123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:50:12.571344 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:50:12.572019 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:50:12.572213 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:50:12.572270 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:50:12.572448 (+ 178us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 03:50:12.572556 (+ 108us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 03:50:12.572613 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0806 03:50:12.572652 (+ 39us) server_negotiation.cc:299] Negotiation successful
0806 03:50:12.572670 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":180,"thread_start_us":126,"threads_started":1}
I20260806 03:50:12.574447 2944 raft_consensus.cc:1261] T 25c0a2833d9e40839c03d7070c39fef6 P fa87c812931045b48ed020a086658673 [term 1 FOLLOWER]: Refusing update from remote peer 3f2f80aff6824a9a9a5601ac2f1732c9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 03:50:12.574446 3127 raft_consensus.cc:1261] T 25c0a2833d9e40839c03d7070c39fef6 P 879486408a1b4379b9154994c41e45e9 [term 1 FOLLOWER]: Refusing update from remote peer 3f2f80aff6824a9a9a5601ac2f1732c9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 03:50:12.574833 3210 consensus_queue.cc:1035] T 25c0a2833d9e40839c03d7070c39fef6 P 3f2f80aff6824a9a9a5601ac2f1732c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "879486408a1b4379b9154994c41e45e9" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 34471 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 03:50:12.574971 2832 consensus_queue.cc:1035] T 25c0a2833d9e40839c03d7070c39fef6 P 3f2f80aff6824a9a9a5601ac2f1732c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa87c812931045b48ed020a086658673" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 34953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 03:50:12.577328 3213 mvcc.cc:205] Tried to move back new op lower bound from 7315407718701998080 to 7315407718352199680. Current Snapshot: MvccSnapshot[applied={T|T < 7315407718701998080}]
I20260806 03:50:12.578418 3218 mvcc.cc:205] Tried to move back new op lower bound from 7315407718701998080 to 7315407718352199680. Current Snapshot: MvccSnapshot[applied={T|T < 7315407718701998080}]
I20260806 03:50:12.579792 2569 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38468:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 03:50:12.579888 2569 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38468:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 03:50:12.580637 2569 catalog_manager.cc:5782] T 00000000000000000000000000000000 P eba5da44f0f04fc6bdc7488532ffbd26: Sending DeleteTablet for 3 replicas of tablet 25c0a2833d9e40839c03d7070c39fef6
I20260806 03:50:12.580929 2734 tablet_service.cc:1514] Processing DeleteTablet for tablet 25c0a2833d9e40839c03d7070c39fef6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 03:50:12 UTC) from {username='kudu', principal='kudu/127.11.253.254@KRBTEST.COM'} at 127.0.0.1:55826
I20260806 03:50:12.580964 2917 tablet_service.cc:1514] Processing DeleteTablet for tablet 25c0a2833d9e40839c03d7070c39fef6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 03:50:12 UTC) from {username='kudu', principal='kudu/127.11.253.254@KRBTEST.COM'} at 127.0.0.1:57534
I20260806 03:50:12.581212 3228 tablet_replica.cc:330] T 25c0a2833d9e40839c03d7070c39fef6 P fa87c812931045b48ed020a086658673: stopping tablet replica
I20260806 03:50:12.581349 12279 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskToVWRe/build/release/bin/kudu with pid 2622
I20260806 03:50:12.581351 3228 raft_consensus.cc:2229] T 25c0a2833d9e40839c03d7070c39fef6 P fa87c812931045b48ed020a086658673 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 03:50:12.586414 3228 raft_consensus.cc:2258] T 25c0a2833d9e40839c03d7070c39fef6 P fa87c812931045b48ed020a086658673 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 03:50:12.589803 3228 ts_tablet_manager.cc:1905] T 25c0a2833d9e40839c03d7070c39fef6 P fa87c812931045b48ed020a086658673: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 03:50:12.590453 3100 tablet_service.cc:1514] Processing DeleteTablet for tablet 25c0a2833d9e40839c03d7070c39fef6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 03:50:12 UTC) from {username='kudu', principal='kudu/127.11.253.254@KRBTEST.COM'} at 127.0.0.1:45356
I20260806 03:50:12.592062 3228 ts_tablet_manager.cc:1918] T 25c0a2833d9e40839c03d7070c39fef6 P fa87c812931045b48ed020a086658673: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 03:50:12.592155 3228 log.cc:1196] T 25c0a2833d9e40839c03d7070c39fef6 P fa87c812931045b48ed020a086658673: Deleting WAL directory at /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/minicluster-data/ts-1/wal/wals/25c0a2833d9e40839c03d7070c39fef6
I20260806 03:50:12.592410 3228 ts_tablet_manager.cc:1939] T 25c0a2833d9e40839c03d7070c39fef6 P fa87c812931045b48ed020a086658673: Deleting consensus metadata
I20260806 03:50:12.592850 2546 catalog_manager.cc:4841] TS fa87c812931045b48ed020a086658673 (127.11.253.194:34953): tablet 25c0a2833d9e40839c03d7070c39fef6 (table test-table [id=feaa80deed054645bb08caffc5a3e567]) successfully deleted
I20260806 03:50:12.599049 3229 tablet_replica.cc:330] T 25c0a2833d9e40839c03d7070c39fef6 P 879486408a1b4379b9154994c41e45e9: stopping tablet replica
I20260806 03:50:12.599218 3229 raft_consensus.cc:2229] T 25c0a2833d9e40839c03d7070c39fef6 P 879486408a1b4379b9154994c41e45e9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 03:50:12.599347 3229 raft_consensus.cc:2258] T 25c0a2833d9e40839c03d7070c39fef6 P 879486408a1b4379b9154994c41e45e9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 03:50:12.599802 3229 ts_tablet_manager.cc:1905] T 25c0a2833d9e40839c03d7070c39fef6 P 879486408a1b4379b9154994c41e45e9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 03:50:12.601377 3229 ts_tablet_manager.cc:1918] T 25c0a2833d9e40839c03d7070c39fef6 P 879486408a1b4379b9154994c41e45e9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 03:50:12.601442 3229 log.cc:1196] T 25c0a2833d9e40839c03d7070c39fef6 P 879486408a1b4379b9154994c41e45e9: Deleting WAL directory at /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/minicluster-data/ts-2/wal/wals/25c0a2833d9e40839c03d7070c39fef6
I20260806 03:50:12.601771 3229 ts_tablet_manager.cc:1939] T 25c0a2833d9e40839c03d7070c39fef6 P 879486408a1b4379b9154994c41e45e9: Deleting consensus metadata
I20260806 03:50:12.602277 2551 catalog_manager.cc:4841] TS 879486408a1b4379b9154994c41e45e9 (127.11.253.195:34471): tablet 25c0a2833d9e40839c03d7070c39fef6 (table test-table [id=feaa80deed054645bb08caffc5a3e567]) successfully deleted
W20260806 03:50:12.604048 2550 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.253.193:42305 (error 108)
I20260806 03:50:12.604352 12279 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskToVWRe/build/release/bin/kudu with pid 2831
I20260806 03:50:12.604565 2834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:50:12.604419 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.193:42305 (local address 127.0.0.1:55840)
0806 03:50:12.604491 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0806 03:50:12.604526 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.253.193:42305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260806 03:50:12.605965 2550 catalog_manager.cc:4584] TS 3f2f80aff6824a9a9a5601ac2f1732c9 (127.11.253.193:42305): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 25c0a2833d9e40839c03d7070c39fef6: Network error: Client connection negotiation failed: client connection to 127.11.253.193:42305: connect: Connection refused (error 111)
I20260806 03:50:12.613986 12279 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskToVWRe/build/release/bin/kudu with pid 3018
I20260806 03:50:12.630868 12279 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskToVWRe/build/release/bin/kudu with pid 2515
I20260806 03:50:32.641105 12279 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskToVWRe/build/release/bin/kudu
/tmp/dist-test-taskToVWRe/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.253.254:42435
--webserver_interface=127.11.253.254
--webserver_port=39741
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.254.keytab
--principal=kudu/127.11.253.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.253.212:40037
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.253.254:42435
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:50:32.729198 3231 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:50:32.729408 3231 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:50:32.729429 3231 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:50:32.730511 3231 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260806 03:50:32.730545 3231 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260806 03:50:32.730558 3231 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260806 03:50:32.730571 3231 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 03:50:32.730583 3231 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:50:32.730597 3231 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:50:32.730608 3231 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 03:50:32.730620 3231 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 03:50:32.732021 3231 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.253.212:40037
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.11.253.254:42435
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.254.keytab
--principal=kudu/127.11.253.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.253.254:42435
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.253.254
--webserver_port=39741
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3231
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:50:32.732205 3231 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:50:32.732393 3231 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:50:32.738035 3241 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:50:32.738137 3238 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:50:32.738034 3237 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:50:32.738219 3231 server_base.cc:1034] running on GCE node
I20260806 03:50:32.738592 3231 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:50:32.738863 3231 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:50:32.740029 3231 hybrid_clock.cc:648] HybridClock initialized: now 1785988232740008 us; error 49 us; skew 500 ppm
Aug 06 03:50:32 dist-test-slave-6ntk krb5kdc[1775](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988232, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.253.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:50:32.742981 3231 init.cc:377] Logged in from keytab as kudu/127.11.253.254@KRBTEST.COM (short username kudu)
I20260806 03:50:32.743433 3231 webserver.cc:460] Webserver started at http://127.11.253.254:39741/ using document root <none> and password file <none>
I20260806 03:50:32.743639 3231 fs_manager.cc:359] Metadata directory not provided
I20260806 03:50:32.743681 3231 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:50:32.745046 3231 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 03:50:32.746033 3252 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:50:32.746222 3231 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 03:50:32.746330 3231 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/minicluster-data/master-0/data,/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/minicluster-data/master-0/wal
uuid: "eba5da44f0f04fc6bdc7488532ffbd26"
format_stamp: "Formatted at 2026-08-06 03:49:50 on dist-test-slave-6ntk"
server_key: "1ca51766568e6569ae46743e4cc146f9"
server_key_iv: "28d770e821969572ffc0c2ff0cfbdb81"
server_key_version: "encryptionkey@0"
I20260806 03:50:32.746685 3231 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:50:32.760283 3231 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:50:32.762437 3231 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:50:32.762562 3231 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:50:32.767222 3231 rpc_server.cc:307] RPC server started. Bound to: 127.11.253.254:42435
I20260806 03:50:32.767258 3325 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.253.254:42435 every 8 connection(s)
I20260806 03:50:32.767681 3231 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/minicluster-data/master-0/data/info.pb
I20260806 03:50:32.768678 12279 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskToVWRe/build/release/bin/kudu as pid 3231
I20260806 03:50:32.769012 12279 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskToVWRe/build/release/bin/kudu
/tmp/dist-test-taskToVWRe/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.253.193:42305
--local_ip_for_outbound_sockets=127.11.253.193
--tserver_master_addrs=127.11.253.254:42435
--webserver_port=44441
--webserver_interface=127.11.253.193
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.193.keytab
--principal=kudu/127.11.253.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.253.212:40037
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 03:50:32.769785 3327 sys_catalog.cc:263] Verifying existing consensus state
I20260806 03:50:32.770982 3327 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eba5da44f0f04fc6bdc7488532ffbd26: Bootstrap starting.
I20260806 03:50:32.780905 3327 log.cc:824] T 00000000000000000000000000000000 P eba5da44f0f04fc6bdc7488532ffbd26: Log is configured to *not* fsync() on all Append() calls
I20260806 03:50:32.783830 3327 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eba5da44f0f04fc6bdc7488532ffbd26: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 03:50:32.784174 3327 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eba5da44f0f04fc6bdc7488532ffbd26: Bootstrap complete.
I20260806 03:50:32.786759 3327 raft_consensus.cc:348] T 00000000000000000000000000000000 P eba5da44f0f04fc6bdc7488532ffbd26 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eba5da44f0f04fc6bdc7488532ffbd26" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 42435 } }
I20260806 03:50:32.786985 3327 raft_consensus.cc:729] T 00000000000000000000000000000000 P eba5da44f0f04fc6bdc7488532ffbd26 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: eba5da44f0f04fc6bdc7488532ffbd26, State: Initialized, Role: FOLLOWER
I20260806 03:50:32.787174 3327 consensus_queue.cc:260] T 00000000000000000000000000000000 P eba5da44f0f04fc6bdc7488532ffbd26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eba5da44f0f04fc6bdc7488532ffbd26" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 42435 } }
I20260806 03:50:32.787240 3327 raft_consensus.cc:388] T 00000000000000000000000000000000 P eba5da44f0f04fc6bdc7488532ffbd26 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 03:50:32.787326 3327 raft_consensus.cc:482] T 00000000000000000000000000000000 P eba5da44f0f04fc6bdc7488532ffbd26 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 03:50:32.787382 3327 raft_consensus.cc:3037] T 00000000000000000000000000000000 P eba5da44f0f04fc6bdc7488532ffbd26 [term 2 FOLLOWER]: Advancing to term 3
I20260806 03:50:32.788439 3327 raft_consensus.cc:504] T 00000000000000000000000000000000 P eba5da44f0f04fc6bdc7488532ffbd26 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eba5da44f0f04fc6bdc7488532ffbd26" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 42435 } }
I20260806 03:50:32.788555 3327 leader_election.cc:302] T 00000000000000000000000000000000 P eba5da44f0f04fc6bdc7488532ffbd26 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: eba5da44f0f04fc6bdc7488532ffbd26; no voters:
I20260806 03:50:32.788774 3327 leader_election.cc:288] T 00000000000000000000000000000000 P eba5da44f0f04fc6bdc7488532ffbd26 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260806 03:50:32.788810 3333 raft_consensus.cc:2781] T 00000000000000000000000000000000 P eba5da44f0f04fc6bdc7488532ffbd26 [term 3 FOLLOWER]: Leader election won for term 3
I20260806 03:50:32.788937 3333 raft_consensus.cc:686] T 00000000000000000000000000000000 P eba5da44f0f04fc6bdc7488532ffbd26 [term 3 LEADER]: Becoming Leader. State: Replica: eba5da44f0f04fc6bdc7488532ffbd26, State: Running, Role: LEADER
I20260806 03:50:32.789080 3327 sys_catalog.cc:564] T 00000000000000000000000000000000 P eba5da44f0f04fc6bdc7488532ffbd26 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 03:50:32.789095 3333 consensus_queue.cc:237] T 00000000000000000000000000000000 P eba5da44f0f04fc6bdc7488532ffbd26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eba5da44f0f04fc6bdc7488532ffbd26" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 42435 } }
I20260806 03:50:32.789598 3335 sys_catalog.cc:455] T 00000000000000000000000000000000 P eba5da44f0f04fc6bdc7488532ffbd26 [sys.catalog]: SysCatalogTable state changed. Reason: New leader eba5da44f0f04fc6bdc7488532ffbd26. Latest consensus state: current_term: 3 leader_uuid: "eba5da44f0f04fc6bdc7488532ffbd26" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eba5da44f0f04fc6bdc7488532ffbd26" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 42435 } } }
I20260806 03:50:32.789693 3335 sys_catalog.cc:458] T 00000000000000000000000000000000 P eba5da44f0f04fc6bdc7488532ffbd26 [sys.catalog]: This master's current role is: LEADER
I20260806 03:50:32.790184 3334 sys_catalog.cc:455] T 00000000000000000000000000000000 P eba5da44f0f04fc6bdc7488532ffbd26 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "eba5da44f0f04fc6bdc7488532ffbd26" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eba5da44f0f04fc6bdc7488532ffbd26" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 42435 } } }
I20260806 03:50:32.790302 3334 sys_catalog.cc:458] T 00000000000000000000000000000000 P eba5da44f0f04fc6bdc7488532ffbd26 [sys.catalog]: This master's current role is: LEADER
I20260806 03:50:32.790441 3342 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 03:50:32.791272 3342 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 03:50:32.791381 3342 catalog_manager.cc:1259] Loaded cluster ID: 93da3e1b34ed419e8b9740addf6b9ad8
I20260806 03:50:32.791415 3342 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 03:50:32.792608 3342 catalog_manager.cc:1504] Loading token signing keys...
I20260806 03:50:32.792806 3342 catalog_manager.cc:5879] T 00000000000000000000000000000000 P eba5da44f0f04fc6bdc7488532ffbd26: Loaded TSK: 0
I20260806 03:50:32.792860 3342 catalog_manager.cc:5879] T 00000000000000000000000000000000 P eba5da44f0f04fc6bdc7488532ffbd26: Loaded TSK: 1
F20260806 03:50:32.793077 3342 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7fdf458649ad google::LogMessage::Fail() at ??:0
@ 0x7fdf4586788a google::LogMessage::SendToLog() at ??:0
@ 0x7fdf458644de google::LogMessage::Flush() at ??:0
@ 0x7fdf45865a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7fdf47fd1104 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7fdf47fced41 kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7fdf4d7be1cd kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7fdf4d7be48e kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7fdf4d7be594 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7fdf4d784c48 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7fdf4d78ead0 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7fdf477320d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7fdf47727954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7fdf4858b6db start_thread at ??:0
@ 0x7fdf4436261f clone at ??:0
W20260806 03:50:32.884466 3329 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:50:32.884797 3329 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:50:32.884846 3329 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:50:32.886674 3329 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:50:32.886745 3329 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:50:32.886796 3329 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.253.193
I20260806 03:50:32.889040 3329 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.253.212:40037
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.193.keytab
--principal=kudu/127.11.253.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.253.193:42305
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.253.193
--webserver_port=44441
--tserver_master_addrs=127.11.253.254:42435
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3329
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.253.193
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:50:32.889431 3329 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:50:32.889731 3329 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:50:32.895547 3383 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:50:32.895731 3386 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:50:32.895578 3384 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:50:32.896075 3329 server_base.cc:1034] running on GCE node
I20260806 03:50:32.896322 3329 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:50:32.896622 3329 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:50:32.897802 3329 hybrid_clock.cc:648] HybridClock initialized: now 1785988232897782 us; error 38 us; skew 500 ppm
Aug 06 03:50:32 dist-test-slave-6ntk krb5kdc[1775](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988232, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.253.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:50:32.900549 3329 init.cc:377] Logged in from keytab as kudu/127.11.253.193@KRBTEST.COM (short username kudu)
I20260806 03:50:32.901031 3329 webserver.cc:460] Webserver started at http://127.11.253.193:44441/ using document root <none> and password file <none>
I20260806 03:50:32.901279 3329 fs_manager.cc:359] Metadata directory not provided
I20260806 03:50:32.901352 3329 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:50:32.902890 3329 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 03:50:32.903690 3397 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:50:32.903932 3329 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 03:50:32.904062 3329 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/minicluster-data/ts-0/data,/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/minicluster-data/ts-0/wal
uuid: "3f2f80aff6824a9a9a5601ac2f1732c9"
format_stamp: "Formatted at 2026-08-06 03:49:51 on dist-test-slave-6ntk"
server_key: "057eb8121ad3c406a7c4008f8251437d"
server_key_iv: "0067d601f4f89d4feec5d3d1d9fa3f33"
server_key_version: "encryptionkey@0"
I20260806 03:50:32.904469 3329 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:50:32.914665 3329 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:50:32.917155 3329 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:50:32.917341 3329 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:50:32.917586 3329 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 03:50:32.918133 3406 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260806 03:50:32.920158 3329 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260806 03:50:32.920280 3329 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260806 03:50:32.920331 3329 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260806 03:50:32.920989 3329 ts_tablet_manager.cc:610] Registered 1 tablets
I20260806 03:50:32.921047 3329 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260806 03:50:32.921128 3406 tablet_bootstrap.cc:492] T 25c0a2833d9e40839c03d7070c39fef6 P 3f2f80aff6824a9a9a5601ac2f1732c9: Bootstrap starting.
I20260806 03:50:32.928563 3329 rpc_server.cc:307] RPC server started. Bound to: 127.11.253.193:42305
I20260806 03:50:32.928661 3551 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.253.193:42305 every 8 connection(s)
I20260806 03:50:32.929034 3329 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/minicluster-data/ts-0/data/info.pb
I20260806 03:50:32.932971 3406 log.cc:824] T 25c0a2833d9e40839c03d7070c39fef6 P 3f2f80aff6824a9a9a5601ac2f1732c9: Log is configured to *not* fsync() on all Append() calls
I20260806 03:50:32.935861 12279 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskToVWRe/build/release/bin/kudu as pid 3329
I20260806 03:50:32.935989 3406 tablet_bootstrap.cc:492] T 25c0a2833d9e40839c03d7070c39fef6 P 3f2f80aff6824a9a9a5601ac2f1732c9: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 03:50:32.936208 12279 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskToVWRe/build/release/bin/kudu
/tmp/dist-test-taskToVWRe/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.253.194:34953
--local_ip_for_outbound_sockets=127.11.253.194
--tserver_master_addrs=127.11.253.254:42435
--webserver_port=39327
--webserver_interface=127.11.253.194
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.194.keytab
--principal=kudu/127.11.253.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.253.212:40037
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 03:50:32.936376 3406 tablet_bootstrap.cc:492] T 25c0a2833d9e40839c03d7070c39fef6 P 3f2f80aff6824a9a9a5601ac2f1732c9: Bootstrap complete.
I20260806 03:50:32.937677 3406 ts_tablet_manager.cc:1397] T 25c0a2833d9e40839c03d7070c39fef6 P 3f2f80aff6824a9a9a5601ac2f1732c9: Time spent bootstrapping tablet: real 0.017s user 0.007s sys 0.005s
I20260806 03:50:32.939447 3406 raft_consensus.cc:348] T 25c0a2833d9e40839c03d7070c39fef6 P 3f2f80aff6824a9a9a5601ac2f1732c9 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa87c812931045b48ed020a086658673" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 34953 } } peers { permanent_uuid: "879486408a1b4379b9154994c41e45e9" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 34471 } } peers { permanent_uuid: "3f2f80aff6824a9a9a5601ac2f1732c9" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 42305 } }
I20260806 03:50:32.939778 3406 raft_consensus.cc:729] T 25c0a2833d9e40839c03d7070c39fef6 P 3f2f80aff6824a9a9a5601ac2f1732c9 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f2f80aff6824a9a9a5601ac2f1732c9, State: Initialized, Role: FOLLOWER
I20260806 03:50:32.939953 3406 consensus_queue.cc:260] T 25c0a2833d9e40839c03d7070c39fef6 P 3f2f80aff6824a9a9a5601ac2f1732c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa87c812931045b48ed020a086658673" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 34953 } } peers { permanent_uuid: "879486408a1b4379b9154994c41e45e9" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 34471 } } peers { permanent_uuid: "3f2f80aff6824a9a9a5601ac2f1732c9" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 42305 } }
I20260806 03:50:32.940253 3406 ts_tablet_manager.cc:1428] T 25c0a2833d9e40839c03d7070c39fef6 P 3f2f80aff6824a9a9a5601ac2f1732c9: Time spent starting tablet: real 0.003s user 0.003s sys 0.003s
W20260806 03:50:33.023638 3557 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 03:50:32.929791 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:42435 (local address 127.11.253.193:33481)
0806 03:50:32.929988 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0806 03:50:32.930030 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 03:50:32.930499 (+ 469us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:50:33.023421 (+ 92922us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.253.254:42435: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":120,"threads_started":1}
W20260806 03:50:33.023968 3552 heartbeater.cc:644] Failed to heartbeat to 127.11.253.254:42435 (0 consecutive failures): Network error: Failed to ping master at 127.11.253.254:42435: Client connection negotiation failed: client connection to 127.11.253.254:42435: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260806 03:50:33.024374 3557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:50:33.024225 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:42435 (local address 127.11.253.193:54069)
0806 03:50:33.024292 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0806 03:50:33.024345 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.253.254:42435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260806 03:50:33.024861 3557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:50:33.024757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:42435 (local address 127.11.253.193:41687)
0806 03:50:33.024806 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0806 03:50:33.024837 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.253.254:42435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260806 03:50:33.025015 3552 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 03:50:33.027941 3560 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:50:33.028131 3560 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:50:33.028154 3560 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:50:33.029321 3560 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:50:33.029361 3560 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:50:33.029388 3560 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.253.194
I20260806 03:50:33.030906 3560 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.253.212:40037
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.194.keytab
--principal=kudu/127.11.253.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.253.194:34953
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.253.194
--webserver_port=39327
--tserver_master_addrs=127.11.253.254:42435
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3560
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.253.194
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:50:33.031154 3560 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:50:33.031391 3560 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:50:33.036509 3568 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:50:33.036477 3572 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:50:33.036492 3560 server_base.cc:1034] running on GCE node
W20260806 03:50:33.036485 3569 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:50:33.036923 3560 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:50:33.037185 3560 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:50:33.038333 3560 hybrid_clock.cc:648] HybridClock initialized: now 1785988233038316 us; error 35 us; skew 500 ppm
Aug 06 03:50:33 dist-test-slave-6ntk krb5kdc[1775](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988233, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.253.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:50:33.041024 3560 init.cc:377] Logged in from keytab as kudu/127.11.253.194@KRBTEST.COM (short username kudu)
I20260806 03:50:33.041460 3560 webserver.cc:460] Webserver started at http://127.11.253.194:39327/ using document root <none> and password file <none>
I20260806 03:50:33.041714 3560 fs_manager.cc:359] Metadata directory not provided
I20260806 03:50:33.041796 3560 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:50:33.043301 3560 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 03:50:33.043987 3581 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:50:33.044225 3560 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 03:50:33.044348 3560 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/minicluster-data/ts-1/data,/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/minicluster-data/ts-1/wal
uuid: "fa87c812931045b48ed020a086658673"
format_stamp: "Formatted at 2026-08-06 03:49:51 on dist-test-slave-6ntk"
server_key: "090c6966b5b8a8ba039f3e35b75630e7"
server_key_iv: "0498c893b2e7d2f6c591c3e1ceb79195"
server_key_version: "encryptionkey@0"
I20260806 03:50:33.044699 3560 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:50:33.061178 3560 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:50:33.063750 3560 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:50:33.063925 3560 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:50:33.064181 3560 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 03:50:33.064497 3560 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 03:50:33.064553 3560 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 03:50:33.064597 3560 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 03:50:33.064622 3560 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 03:50:33.072392 3560 rpc_server.cc:307] RPC server started. Bound to: 127.11.253.194:34953
I20260806 03:50:33.072432 3726 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.253.194:34953 every 8 connection(s)
I20260806 03:50:33.072817 3560 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/minicluster-data/ts-1/data/info.pb
I20260806 03:50:33.074083 3730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:50:33.073549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:42435 (local address 127.11.253.194:38683)
0806 03:50:33.073757 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0806 03:50:33.073843 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.253.254:42435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":75,"threads_started":1}
W20260806 03:50:33.074544 3728 heartbeater.cc:644] Failed to heartbeat to 127.11.253.254:42435 (0 consecutive failures): Network error: Failed to ping master at 127.11.253.254:42435: Client connection negotiation failed: client connection to 127.11.253.254:42435: connect: Connection refused (error 111)
I20260806 03:50:33.074839 3730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:50:33.074738 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:42435 (local address 127.11.253.194:56751)
0806 03:50:33.074788 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0806 03:50:33.074820 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.253.254:42435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260806 03:50:33.075084 3730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:50:33.075029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:42435 (local address 127.11.253.194:40091)
0806 03:50:33.075052 (+ 23us) negotiation.cc:107] Waiting for socket to connect
0806 03:50:33.075072 (+ 20us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.253.254:42435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":13}
W20260806 03:50:33.075189 3728 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260806 03:50:33.079180 12279 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskToVWRe/build/release/bin/kudu as pid 3560
I20260806 03:50:33.079488 12279 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskToVWRe/build/release/bin/kudu
/tmp/dist-test-taskToVWRe/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.253.195:34471
--local_ip_for_outbound_sockets=127.11.253.195
--tserver_master_addrs=127.11.253.254:42435
--webserver_port=43257
--webserver_interface=127.11.253.195
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.195.keytab
--principal=kudu/127.11.253.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.253.212:40037
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:50:33.165037 3731 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:50:33.165305 3731 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:50:33.165360 3731 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:50:33.166638 3731 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:50:33.166707 3731 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:50:33.166751 3731 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.253.195
I20260806 03:50:33.168226 3731 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.253.212:40037
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.195.keytab
--principal=kudu/127.11.253.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.253.195:34471
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.253.195
--webserver_port=43257
--tserver_master_addrs=127.11.253.254:42435
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3731
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.253.195
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:50:33.168501 3731 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:50:33.168746 3731 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:50:33.174041 3741 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:50:33.174088 3736 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:50:33.174036 3737 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:50:33.174074 3731 server_base.cc:1034] running on GCE node
I20260806 03:50:33.174494 3731 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:50:33.174690 3731 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:50:33.175834 3731 hybrid_clock.cc:648] HybridClock initialized: now 1785988233175822 us; error 38 us; skew 500 ppm
Aug 06 03:50:33 dist-test-slave-6ntk krb5kdc[1775](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988233, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.253.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:50:33.178870 3731 init.cc:377] Logged in from keytab as kudu/127.11.253.195@KRBTEST.COM (short username kudu)
I20260806 03:50:33.179409 3731 webserver.cc:460] Webserver started at http://127.11.253.195:43257/ using document root <none> and password file <none>
I20260806 03:50:33.179677 3731 fs_manager.cc:359] Metadata directory not provided
I20260806 03:50:33.179734 3731 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:50:33.181177 3731 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 03:50:33.181985 3751 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:50:33.182197 3731 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 03:50:33.182305 3731 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/minicluster-data/ts-2/data,/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/minicluster-data/ts-2/wal
uuid: "879486408a1b4379b9154994c41e45e9"
format_stamp: "Formatted at 2026-08-06 03:49:51 on dist-test-slave-6ntk"
server_key: "29100263602bca389eb0279894831131"
server_key_iv: "e8d05ae3aaccbc0c2abdd6af0c30829a"
server_key_version: "encryptionkey@0"
I20260806 03:50:33.182651 3731 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:50:33.201594 3731 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:50:33.204363 3731 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:50:33.204563 3731 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:50:33.204907 3731 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 03:50:33.205341 3731 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 03:50:33.205396 3731 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 03:50:33.205438 3731 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 03:50:33.205484 3731 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 03:50:33.214134 3731 rpc_server.cc:307] RPC server started. Bound to: 127.11.253.195:34471
I20260806 03:50:33.214179 3913 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.253.195:34471 every 8 connection(s)
I20260806 03:50:33.214565 3731 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988004637995-12279-0/minicluster-data/ts-2/data/info.pb
I20260806 03:50:33.215885 3916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:50:33.215328 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:42435 (local address 127.11.253.195:57545)
0806 03:50:33.215515 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0806 03:50:33.215621 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.253.254:42435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":78,"threads_started":1}
I20260806 03:50:33.216061 12279 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskToVWRe/build/release/bin/kudu as pid 3731
W20260806 03:50:33.216429 3914 heartbeater.cc:644] Failed to heartbeat to 127.11.253.254:42435 (0 consecutive failures): Network error: Failed to ping master at 127.11.253.254:42435: Client connection negotiation failed: client connection to 127.11.253.254:42435: connect: Connection refused (error 111)
I20260806 03:50:33.216868 3919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:50:33.216652 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:42435 (local address 127.11.253.195:42235)
0806 03:50:33.216790 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0806 03:50:33.216825 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.253.254:42435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":78,"threads_started":1}
I20260806 03:50:33.217188 3919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:50:33.217110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:42435 (local address 127.11.253.195:33353)
0806 03:50:33.217153 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0806 03:50:33.217173 (+ 20us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.253.254:42435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260806 03:50:33.217374 3914 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260806 03:50:34.025704 3920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:50:34.025450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:42435 (local address 127.11.253.193:40417)
0806 03:50:34.025597 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0806 03:50:34.025636 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.253.254:42435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":71,"threads_started":1}
I20260806 03:50:34.075994 3922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:50:34.075736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:42435 (local address 127.11.253.194:33295)
0806 03:50:34.075899 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0806 03:50:34.075944 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.253.254:42435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":100,"threads_started":1}
I20260806 03:50:34.218034 3923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:50:34.217701 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:42435 (local address 127.11.253.195:52713)
0806 03:50:34.217860 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0806 03:50:34.217972 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.253.254:42435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":64,"threads_started":1}
I20260806 03:50:34.792470 3925 raft_consensus.cc:482] T 25c0a2833d9e40839c03d7070c39fef6 P 3f2f80aff6824a9a9a5601ac2f1732c9 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 03:50:34.792660 3925 raft_consensus.cc:504] T 25c0a2833d9e40839c03d7070c39fef6 P 3f2f80aff6824a9a9a5601ac2f1732c9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa87c812931045b48ed020a086658673" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 34953 } } peers { permanent_uuid: "879486408a1b4379b9154994c41e45e9" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 34471 } } peers { permanent_uuid: "3f2f80aff6824a9a9a5601ac2f1732c9" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 42305 } }
I20260806 03:50:34.793084 3925 leader_election.cc:288] T 25c0a2833d9e40839c03d7070c39fef6 P 3f2f80aff6824a9a9a5601ac2f1732c9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fa87c812931045b48ed020a086658673 (127.11.253.194:34953), 879486408a1b4379b9154994c41e45e9 (127.11.253.195:34471)
Aug 06 03:50:34 dist-test-slave-6ntk krb5kdc[1775](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988232, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.253.193@KRBTEST.COM for kudu/127.11.253.194@KRBTEST.COM
Aug 06 03:50:34 dist-test-slave-6ntk krb5kdc[1775](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988232, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.253.193@KRBTEST.COM for kudu/127.11.253.195@KRBTEST.COM
I20260806 03:50:34.804373 3933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:50:34.793490 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.253.193:50847 (local address 127.11.253.194:34953)
0806 03:50:34.793696 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0806 03:50:34.793707 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 03:50:34.793728 (+ 21us) server_negotiation.cc:407] Connection header received
0806 03:50:34.794235 (+ 507us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:50:34.794239 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:50:34.794287 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:50:34.794436 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:50:34.795261 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:50:34.796390 (+ 1129us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:50:34.797406 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:50:34.797844 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:50:34.798012 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:50:34.800503 (+ 2491us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:50:34.800507 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:50:34.800507 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:50:34.800538 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:50:34.802671 (+ 2133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:50:34.803126 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:50:34.803129 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:50:34.803130 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:50:34.803221 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:50:34.803491 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:50:34.803492 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:50:34.803492 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:50:34.803742 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:50:34.803839 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0806 03:50:34.804131 (+ 292us) server_negotiation.cc:299] Negotiation successful
0806 03:50:34.804259 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":169,"thread_start_us":82,"threads_started":1}
I20260806 03:50:34.804455 3929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:50:34.793290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.194:34953 (local address 127.11.253.193:50847)
0806 03:50:34.793481 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0806 03:50:34.793494 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 03:50:34.793563 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:50:34.794474 (+ 911us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:50:34.794477 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:50:34.794910 (+ 433us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:50:34.795138 (+ 228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:50:34.795145 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:50:34.796489 (+ 1344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:50:34.796491 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:50:34.797310 (+ 819us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:50:34.797312 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:50:34.797980 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:50:34.797981 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:50:34.798094 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:50:34.798839 (+ 745us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:50:34.798843 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:50:34.800379 (+ 1536us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:50:34.802773 (+ 2394us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:50:34.802781 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:50:34.802783 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:50:34.803012 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:50:34.803313 (+ 301us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:50:34.803313 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:50:34.803314 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:50:34.803413 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:50:34.803834 (+ 421us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:50:34.803841 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:50:34.803963 (+ 122us) client_negotiation.cc:769] Sending connection context
0806 03:50:34.804073 (+ 110us) client_negotiation.cc:240] Negotiation successful
0806 03:50:34.804327 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":173,"mutex_wait_us":47,"thread_start_us":56,"threads_started":1}
I20260806 03:50:34.804507 3930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:50:34.793349 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.195:34471 (local address 127.11.253.193:51559)
0806 03:50:34.793495 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0806 03:50:34.793508 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 03:50:34.793570 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:50:34.794474 (+ 904us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:50:34.794477 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:50:34.794909 (+ 432us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:50:34.795139 (+ 230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:50:34.795146 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:50:34.796481 (+ 1335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:50:34.796482 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:50:34.797308 (+ 826us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:50:34.797310 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:50:34.797956 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:50:34.797957 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:50:34.798099 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:50:34.798790 (+ 691us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:50:34.798799 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:50:34.800765 (+ 1966us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:50:34.802999 (+ 2234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:50:34.803000 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:50:34.803000 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:50:34.803206 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:50:34.803480 (+ 274us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:50:34.803480 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:50:34.803481 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:50:34.803547 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:50:34.803948 (+ 401us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:50:34.803949 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:50:34.803998 (+ 49us) client_negotiation.cc:769] Sending connection context
0806 03:50:34.804089 (+ 91us) client_negotiation.cc:240] Negotiation successful
0806 03:50:34.804357 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"mutex_wait_us":153,"thread_start_us":55,"threads_started":1}
I20260806 03:50:34.804435 3932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:50:34.793496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.253.193:51559 (local address 127.11.253.195:34471)
0806 03:50:34.793709 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0806 03:50:34.793718 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 03:50:34.793738 (+ 20us) server_negotiation.cc:407] Connection header received
0806 03:50:34.794242 (+ 504us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:50:34.794244 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:50:34.794287 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:50:34.794436 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:50:34.795263 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:50:34.796383 (+ 1120us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:50:34.797402 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:50:34.797824 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:50:34.797992 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:50:34.800874 (+ 2882us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:50:34.800879 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:50:34.800879 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:50:34.800892 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:50:34.802917 (+ 2025us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:50:34.803295 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:50:34.803299 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:50:34.803299 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:50:34.803402 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:50:34.803613 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:50:34.803614 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:50:34.803615 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:50:34.803863 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:50:34.803951 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0806 03:50:34.804156 (+ 205us) server_negotiation.cc:299] Negotiation successful
0806 03:50:34.804314 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":180,"thread_start_us":60,"threads_started":1}
W20260806 03:50:34.805514 3400 leader_election.cc:341] T 25c0a2833d9e40839c03d7070c39fef6 P 3f2f80aff6824a9a9a5601ac2f1732c9 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer fa87c812931045b48ed020a086658673 (127.11.253.194:34953): Not found: Tablet not found: 25c0a2833d9e40839c03d7070c39fef6
W20260806 03:50:34.805610 3403 leader_election.cc:341] T 25c0a2833d9e40839c03d7070c39fef6 P 3f2f80aff6824a9a9a5601ac2f1732c9 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 879486408a1b4379b9154994c41e45e9 (127.11.253.195:34471): Not found: Tablet not found: 25c0a2833d9e40839c03d7070c39fef6
I20260806 03:50:34.805667 3403 leader_election.cc:302] T 25c0a2833d9e40839c03d7070c39fef6 P 3f2f80aff6824a9a9a5601ac2f1732c9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f2f80aff6824a9a9a5601ac2f1732c9; no voters: 879486408a1b4379b9154994c41e45e9, fa87c812931045b48ed020a086658673
I20260806 03:50:34.805807 3925 raft_consensus.cc:2726] T 25c0a2833d9e40839c03d7070c39fef6 P 3f2f80aff6824a9a9a5601ac2f1732c9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 03:50:35.026412 3929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:50:35.026238 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:42435 (local address 127.11.253.193:36733)
0806 03:50:35.026309 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0806 03:50:35.026364 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.253.254:42435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260806 03:50:35.076798 3935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:50:35.076527 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:42435 (local address 127.11.253.194:34859)
0806 03:50:35.076679 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0806 03:50:35.076725 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.253.254:42435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":65,"threads_started":1}
I20260806 03:50:35.218873 3936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:50:35.218565 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:42435 (local address 127.11.253.195:33325)
0806 03:50:35.218753 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0806 03:50:35.218799 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.253.254:42435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":92,"threads_started":1}
I20260806 03:50:36.027340 3937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:50:36.026967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:42435 (local address 127.11.253.193:36975)
0806 03:50:36.027245 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0806 03:50:36.027287 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.253.254:42435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":259,"thread_start_us":87,"threads_started":1}
I20260806 03:50:36.077670 3938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:50:36.077422 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:42435 (local address 127.11.253.194:57707)
0806 03:50:36.077576 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0806 03:50:36.077623 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.253.254:42435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":74,"threads_started":1}
I20260806 03:50:36.219661 3939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:50:36.219421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:42435 (local address 127.11.253.195:49755)
0806 03:50:36.219573 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0806 03:50:36.219608 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.253.254:42435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":91,"threads_started":1}
I20260806 03:50:36.841111 3941 raft_consensus.cc:482] T 25c0a2833d9e40839c03d7070c39fef6 P 3f2f80aff6824a9a9a5601ac2f1732c9 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 03:50:36.841245 3941 raft_consensus.cc:504] T 25c0a2833d9e40839c03d7070c39fef6 P 3f2f80aff6824a9a9a5601ac2f1732c9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa87c812931045b48ed020a086658673" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 34953 } } peers { permanent_uuid: "879486408a1b4379b9154994c41e45e9" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 34471 } } peers { permanent_uuid: "3f2f80aff6824a9a9a5601ac2f1732c9" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 42305 } }
I20260806 03:50:36.841495 3941 leader_election.cc:288] T 25c0a2833d9e40839c03d7070c39fef6 P 3f2f80aff6824a9a9a5601ac2f1732c9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fa87c812931045b48ed020a086658673 (127.11.253.194:34953), 879486408a1b4379b9154994c41e45e9 (127.11.253.195:34471)
W20260806 03:50:36.841992 3403 leader_election.cc:341] T 25c0a2833d9e40839c03d7070c39fef6 P 3f2f80aff6824a9a9a5601ac2f1732c9 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 879486408a1b4379b9154994c41e45e9 (127.11.253.195:34471): Not found: Tablet not found: 25c0a2833d9e40839c03d7070c39fef6
W20260806 03:50:36.842077 3400 leader_election.cc:341] T 25c0a2833d9e40839c03d7070c39fef6 P 3f2f80aff6824a9a9a5601ac2f1732c9 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer fa87c812931045b48ed020a086658673 (127.11.253.194:34953): Not found: Tablet not found: 25c0a2833d9e40839c03d7070c39fef6
I20260806 03:50:36.842118 3400 leader_election.cc:302] T 25c0a2833d9e40839c03d7070c39fef6 P 3f2f80aff6824a9a9a5601ac2f1732c9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f2f80aff6824a9a9a5601ac2f1732c9; no voters: 879486408a1b4379b9154994c41e45e9, fa87c812931045b48ed020a086658673
I20260806 03:50:36.842195 3941 raft_consensus.cc:2726] T 25c0a2833d9e40839c03d7070c39fef6 P 3f2f80aff6824a9a9a5601ac2f1732c9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 03:50:37.028177 3942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:50:37.027929 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:42435 (local address 127.11.253.193:48197)
0806 03:50:37.028066 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0806 03:50:37.028106 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.253.254:42435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":65,"threads_started":1}
I20260806 03:50:37.078517 3944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:50:37.078266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:42435 (local address 127.11.253.194:53875)
0806 03:50:37.078422 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0806 03:50:37.078466 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.253.254:42435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":72,"threads_started":1}
I20260806 03:50:37.220557 3945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:50:37.220283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:42435 (local address 127.11.253.195:48969)
0806 03:50:37.220450 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0806 03:50:37.220491 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.253.254:42435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":71,"threads_started":1}
I20260806 03:50:38.029126 3946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:50:38.028825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:42435 (local address 127.11.253.193:56549)
0806 03:50:38.029028 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0806 03:50:38.029077 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.253.254:42435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":105,"threads_started":1}
I20260806 03:50:38.079269 3947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:50:38.079055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:42435 (local address 127.11.253.194:56287)
0806 03:50:38.079186 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0806 03:50:38.079225 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.253.254:42435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":67,"threads_started":1}
I20260806 03:50:38.221360 3948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:50:38.221113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:42435 (local address 127.11.253.195:48077)
0806 03:50:38.221259 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0806 03:50:38.221302 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.253.254:42435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":83,"threads_started":1}
I20260806 03:50:39.030053 3950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:50:39.029739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:42435 (local address 127.11.253.193:41299)
0806 03:50:39.029928 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0806 03:50:39.029994 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.253.254:42435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":102,"threads_started":1}
I20260806 03:50:39.036121 3952 raft_consensus.cc:482] T 25c0a2833d9e40839c03d7070c39fef6 P 3f2f80aff6824a9a9a5601ac2f1732c9 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 03:50:39.036221 3952 raft_consensus.cc:504] T 25c0a2833d9e40839c03d7070c39fef6 P 3f2f80aff6824a9a9a5601ac2f1732c9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa87c812931045b48ed020a086658673" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 34953 } } peers { permanent_uuid: "879486408a1b4379b9154994c41e45e9" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 34471 } } peers { permanent_uuid: "3f2f80aff6824a9a9a5601ac2f1732c9" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 42305 } }
I20260806 03:50:39.036473 3952 leader_election.cc:288] T 25c0a2833d9e40839c03d7070c39fef6 P 3f2f80aff6824a9a9a5601ac2f1732c9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fa87c812931045b48ed020a086658673 (127.11.253.194:34953), 879486408a1b4379b9154994c41e45e9 (127.11.253.195:34471)
W20260806 03:50:39.036872 3400 leader_election.cc:341] T 25c0a2833d9e40839c03d7070c39fef6 P 3f2f80aff6824a9a9a5601ac2f1732c9 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer fa87c812931045b48ed020a086658673 (127.11.253.194:34953): Not found: Tablet not found: 25c0a2833d9e40839c03d7070c39fef6
W20260806 03:50:39.036988 3403 leader_election.cc:341] T 25c0a2833d9e40839c03d7070c39fef6 P 3f2f80aff6824a9a9a5601ac2f1732c9 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 879486408a1b4379b9154994c41e45e9 (127.11.253.195:34471): Not found: Tablet not found: 25c0a2833d9e40839c03d7070c39fef6
I20260806 03:50:39.037066 3403 leader_election.cc:302] T 25c0a2833d9e40839c03d7070c39fef6 P 3f2f80aff6824a9a9a5601ac2f1732c9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f2f80aff6824a9a9a5601ac2f1732c9; no voters: 879486408a1b4379b9154994c41e45e9, fa87c812931045b48ed020a086658673
I20260806 03:50:39.037163 3952 raft_consensus.cc:2726] T 25c0a2833d9e40839c03d7070c39fef6 P 3f2f80aff6824a9a9a5601ac2f1732c9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 03:50:39.080086 3953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:50:39.079808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:42435 (local address 127.11.253.194:60511)
0806 03:50:39.079993 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0806 03:50:39.080039 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.253.254:42435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":76,"threads_started":1}
I20260806 03:50:39.222204 3954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:50:39.221987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:42435 (local address 127.11.253.195:49133)
0806 03:50:39.222114 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0806 03:50:39.222152 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.253.254:42435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":62,"threads_started":1}
I20260806 03:50:40.030800 3955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:50:40.030573 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:42435 (local address 127.11.253.193:48787)
0806 03:50:40.030703 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0806 03:50:40.030747 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.253.254:42435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":68,"threads_started":1}
I20260806 03:50:40.080912 3956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:50:40.080617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:42435 (local address 127.11.253.194:45989)
0806 03:50:40.080814 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0806 03:50:40.080863 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.253.254:42435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":63,"threads_started":1}
I20260806 03:50:40.223052 3957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:50:40.222772 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:42435 (local address 127.11.253.195:38377)
0806 03:50:40.222950 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0806 03:50:40.222998 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.253.254:42435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":71,"threads_started":1}
I20260806 03:50:41.031590 3959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:50:41.031309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:42435 (local address 127.11.253.193:42195)
0806 03:50:41.031482 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0806 03:50:41.031529 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.253.254:42435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":69,"threads_started":1}
I20260806 03:50:41.081764 3961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:50:41.081534 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:42435 (local address 127.11.253.194:43699)
0806 03:50:41.081671 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0806 03:50:41.081716 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.253.254:42435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":71,"threads_started":1}
I20260806 03:50:41.223815 3963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:50:41.223564 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:42435 (local address 127.11.253.195:47131)
0806 03:50:41.223717 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0806 03:50:41.223756 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.253.254:42435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":76,"threads_started":1}
I20260806 03:50:42.032598 3969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:50:42.032300 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:42435 (local address 127.11.253.193:51677)
0806 03:50:42.032490 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0806 03:50:42.032543 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.253.254:42435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":83,"threads_started":1}
I20260806 03:50:42.082544 3970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:50:42.082316 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:42435 (local address 127.11.253.194:46915)
0806 03:50:42.082452 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0806 03:50:42.082495 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.253.254:42435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":64,"threads_started":1}
I20260806 03:50:42.224735 3971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:50:42.224464 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:42435 (local address 127.11.253.195:38005)
0806 03:50:42.224627 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0806 03:50:42.224675 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.253.254:42435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":89,"threads_started":1}
I20260806 03:50:42.640164 3972 raft_consensus.cc:482] T 25c0a2833d9e40839c03d7070c39fef6 P 3f2f80aff6824a9a9a5601ac2f1732c9 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 03:50:42.640265 3972 raft_consensus.cc:504] T 25c0a2833d9e40839c03d7070c39fef6 P 3f2f80aff6824a9a9a5601ac2f1732c9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa87c812931045b48ed020a086658673" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 34953 } } peers { permanent_uuid: "879486408a1b4379b9154994c41e45e9" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 34471 } } peers { permanent_uuid: "3f2f80aff6824a9a9a5601ac2f1732c9" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 42305 } }
I20260806 03:50:42.640506 3972 leader_election.cc:288] T 25c0a2833d9e40839c03d7070c39fef6 P 3f2f80aff6824a9a9a5601ac2f1732c9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fa87c812931045b48ed020a086658673 (127.11.253.194:34953), 879486408a1b4379b9154994c41e45e9 (127.11.253.195:34471)
W20260806 03:50:42.640878 3400 leader_election.cc:341] T 25c0a2833d9e40839c03d7070c39fef6 P 3f2f80aff6824a9a9a5601ac2f1732c9 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer fa87c812931045b48ed020a086658673 (127.11.253.194:34953): Not found: Tablet not found: 25c0a2833d9e40839c03d7070c39fef6
W20260806 03:50:42.640973 3403 leader_election.cc:341] T 25c0a2833d9e40839c03d7070c39fef6 P 3f2f80aff6824a9a9a5601ac2f1732c9 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 879486408a1b4379b9154994c41e45e9 (127.11.253.195:34471): Not found: Tablet not found: 25c0a2833d9e40839c03d7070c39fef6
I20260806 03:50:42.641043 3403 leader_election.cc:302] T 25c0a2833d9e40839c03d7070c39fef6 P 3f2f80aff6824a9a9a5601ac2f1732c9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f2f80aff6824a9a9a5601ac2f1732c9; no voters: 879486408a1b4379b9154994c41e45e9, fa87c812931045b48ed020a086658673
I20260806 03:50:42.641129 3972 raft_consensus.cc:2726] T 25c0a2833d9e40839c03d7070c39fef6 P 3f2f80aff6824a9a9a5601ac2f1732c9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 03:50:43.033473 3973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:50:43.033190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:42435 (local address 127.11.253.193:52887)
0806 03:50:43.033368 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0806 03:50:43.033415 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.253.254:42435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":86,"threads_started":1}
I20260806 03:50:43.083408 3974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:50:43.083106 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:42435 (local address 127.11.253.194:50579)
0806 03:50:43.083312 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0806 03:50:43.083359 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.253.254:42435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":74,"threads_started":1}
I20260806 03:50:43.225540 3975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:50:43.225301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:42435 (local address 127.11.253.195:59697)
0806 03:50:43.225446 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0806 03:50:43.225487 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.253.254:42435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":77,"threads_started":1}
I20260806 03:50:44.034474 3976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:50:44.034219 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:42435 (local address 127.11.253.193:49529)
0806 03:50:44.034372 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0806 03:50:44.034421 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.253.254:42435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":77,"threads_started":1}
I20260806 03:50:44.084169 3977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:50:44.083952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:42435 (local address 127.11.253.194:51721)
0806 03:50:44.084081 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0806 03:50:44.084119 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.253.254:42435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":64,"threads_started":1}
I20260806 03:50:44.226450 3978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:50:44.226158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:42435 (local address 127.11.253.195:48913)
0806 03:50:44.226354 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0806 03:50:44.226396 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.253.254:42435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":80,"threads_started":1}
I20260806 03:50:45.035324 3980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:50:45.035046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:42435 (local address 127.11.253.193:47801)
0806 03:50:45.035188 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0806 03:50:45.035271 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.253.254:42435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":76,"threads_started":1}
I20260806 03:50:45.085031 3982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:50:45.084670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:42435 (local address 127.11.253.194:37603)
0806 03:50:45.084861 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0806 03:50:45.084960 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.253.254:42435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":118,"threads_started":1}
I20260806 03:50:45.227234 3985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:50:45.226982 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:42435 (local address 127.11.253.195:49637)
0806 03:50:45.227135 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0806 03:50:45.227176 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.253.254:42435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":75,"threads_started":1}
I20260806 03:50:46.036229 3986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:50:46.035918 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:42435 (local address 127.11.253.193:58507)
0806 03:50:46.036127 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0806 03:50:46.036176 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.253.254:42435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":98,"threads_started":1}
I20260806 03:50:46.085804 3987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:50:46.085552 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:42435 (local address 127.11.253.194:42633)
0806 03:50:46.085681 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0806 03:50:46.085757 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.253.254:42435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":67,"threads_started":1}
I20260806 03:50:46.228076 3988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:50:46.227732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:42435 (local address 127.11.253.195:52279)
0806 03:50:46.227937 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0806 03:50:46.227985 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.253.254:42435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":110,"threads_started":1}
I20260806 03:50:47.037191 3989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:50:47.036913 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:42435 (local address 127.11.253.193:47707)
0806 03:50:47.037089 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0806 03:50:47.037138 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.253.254:42435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":85,"threads_started":1}
I20260806 03:50:47.086668 3990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:50:47.086427 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:42435 (local address 127.11.253.194:35199)
0806 03:50:47.086570 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0806 03:50:47.086617 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.253.254:42435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":65,"threads_started":1}
I20260806 03:50:47.228789 3991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:50:47.228565 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:42435 (local address 127.11.253.195:52539)
0806 03:50:47.228706 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0806 03:50:47.228738 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.253.254:42435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":83,"threads_started":1}
I20260806 03:50:48.038308 3994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:50:48.038064 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:42435 (local address 127.11.253.193:49635)
0806 03:50:48.038215 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0806 03:50:48.038257 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.253.254:42435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":67,"threads_started":1}
I20260806 03:50:48.087505 3995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:50:48.087286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:42435 (local address 127.11.253.194:37747)
0806 03:50:48.087421 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0806 03:50:48.087458 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.253.254:42435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":69,"threads_started":1}
I20260806 03:50:48.221227 12279 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskToVWRe/build/release/bin/kudu with pid 3329
I20260806 03:50:48.229664 3997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:50:48.229423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:42435 (local address 127.11.253.195:37603)
0806 03:50:48.229572 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0806 03:50:48.229617 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.253.254:42435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":82,"threads_started":1}
I20260806 03:50:48.233688 12279 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskToVWRe/build/release/bin/kudu with pid 3560
I20260806 03:50:48.246596 12279 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskToVWRe/build/release/bin/kudu with pid 3731
2026-08-06T03:50:48Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (58892 ms)
[----------] 30 tests from SecurityITest (243620 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785988004637995-12279-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 03:50:48 dist-test-slave-6ntk krb5kdc[4000](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 03:50:48 dist-test-slave-6ntk krb5kdc[4000](info): set up 2 sockets
Aug 06 03:50:48 dist-test-slave-6ntk krb5kdc[4000](info): commencing operation
krb5kdc: starting...
W20260806 03:50:49.498394 12279 mini_kdc.cc:121] Time spent starting KDC: real 1.225s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785988004637995-12279-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785988004637995-12279-0/krb5kdc/test-user.keytab.
Aug 06 03:50:49 dist-test-slave-6ntk krb5kdc[4000](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988249, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T03:50:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T03:50:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.253.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.253.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.253.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.254.keytab.
Entry for principal kudu/127.11.253.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.254.keytab.
I20260806 03:50:49.630182 12279 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskToVWRe/build/release/bin/kudu
/tmp/dist-test-taskToVWRe/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785988004637995-12279-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785988004637995-12279-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785988004637995-12279-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785988004637995-12279-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.253.254:42665
--webserver_interface=127.11.253.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.254.keytab
--principal=kudu/127.11.253.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.253.212:45783
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.253.254:42665
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785988004637995-12279-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785988004637995-12279-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785988004637995-12279-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:50:49.713845 4018 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:50:49.714108 4018 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:50:49.714131 4018 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:50:49.715368 4018 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 03:50:49.715412 4018 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:50:49.715426 4018 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:50:49.715440 4018 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 03:50:49.715494 4018 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 03:50:49.716985 4018 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.253.212:45783
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785988004637995-12279-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785988004637995-12279-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.253.254:42665
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.254.keytab
--principal=kudu/127.11.253.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.253.254:42665
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785988004637995-12279-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.253.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4018
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785988004637995-12279-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:50:49.717252 4018 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:50:49.717499 4018 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:50:49.722791 4028 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:50:49.722841 4026 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:50:49.722826 4025 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:50:49.722858 4018 server_base.cc:1034] running on GCE node
I20260806 03:50:49.723363 4018 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:50:49.723613 4018 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:50:49.724778 4018 hybrid_clock.cc:648] HybridClock initialized: now 1785988249724765 us; error 46 us; skew 500 ppm
Aug 06 03:50:49 dist-test-slave-6ntk krb5kdc[4000](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988249, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.253.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:50:49.727360 4018 init.cc:377] Logged in from keytab as kudu/127.11.253.254@KRBTEST.COM (short username kudu)
I20260806 03:50:49.727865 4018 webserver.cc:460] Webserver started at http://127.11.253.254:41747/ using document root <none> and password file <none>
I20260806 03:50:49.728098 4018 fs_manager.cc:359] Metadata directory not provided
I20260806 03:50:49.728168 4018 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:50:49.728328 4018 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:50:49.729396 4018 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785988004637995-12279-0/minicluster-data/master-0/data/instance:
uuid: "2fc28baac5404615b5abc35eb19db6ef"
format_stamp: "Formatted at 2026-08-06 03:50:49 on dist-test-slave-6ntk"
server_key: "021ec4a65cdda68dbc347b807f5f7e3f"
server_key_iv: "eb2cd5a18335fc05199c55008f362530"
server_key_version: "encryptionkey@0"
I20260806 03:50:49.729771 4018 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785988004637995-12279-0/minicluster-data/master-0/wal/instance:
uuid: "2fc28baac5404615b5abc35eb19db6ef"
format_stamp: "Formatted at 2026-08-06 03:50:49 on dist-test-slave-6ntk"
server_key: "021ec4a65cdda68dbc347b807f5f7e3f"
server_key_iv: "eb2cd5a18335fc05199c55008f362530"
server_key_version: "encryptionkey@0"
I20260806 03:50:49.731208 4018 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 03:50:49.732038 4037 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:50:49.732324 4018 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 03:50:49.732470 4018 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785988004637995-12279-0/minicluster-data/master-0/data,/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785988004637995-12279-0/minicluster-data/master-0/wal
uuid: "2fc28baac5404615b5abc35eb19db6ef"
format_stamp: "Formatted at 2026-08-06 03:50:49 on dist-test-slave-6ntk"
server_key: "021ec4a65cdda68dbc347b807f5f7e3f"
server_key_iv: "eb2cd5a18335fc05199c55008f362530"
server_key_version: "encryptionkey@0"
I20260806 03:50:49.732556 4018 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785988004637995-12279-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785988004637995-12279-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785988004637995-12279-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:50:49.754562 4018 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:50:49.756695 4018 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:50:49.756870 4018 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:50:49.761415 4018 rpc_server.cc:307] RPC server started. Bound to: 127.11.253.254:42665
I20260806 03:50:49.761495 4116 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.253.254:42665 every 8 connection(s)
I20260806 03:50:49.761883 4018 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785988004637995-12279-0/minicluster-data/master-0/data/info.pb
I20260806 03:50:49.762777 4117 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:50:49.765031 4117 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2fc28baac5404615b5abc35eb19db6ef: Bootstrap starting.
I20260806 03:50:49.765435 12279 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskToVWRe/build/release/bin/kudu as pid 4018
I20260806 03:50:49.765553 12279 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785988004637995-12279-0/minicluster-data/master-0/wal/instance
I20260806 03:50:49.765650 4117 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2fc28baac5404615b5abc35eb19db6ef: Neither blocks nor log segments found. Creating new log.
I20260806 03:50:49.765789 12279 external_mini_cluster.cc:1428] Setting key 2834ee8c76f78ca7961e51aa55755415
I20260806 03:50:49.766076 4117 log.cc:824] T 00000000000000000000000000000000 P 2fc28baac5404615b5abc35eb19db6ef: Log is configured to *not* fsync() on all Append() calls
I20260806 03:50:49.766908 4117 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2fc28baac5404615b5abc35eb19db6ef: No bootstrap required, opened a new log
I20260806 03:50:49.768863 4117 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2fc28baac5404615b5abc35eb19db6ef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2fc28baac5404615b5abc35eb19db6ef" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 42665 } }
I20260806 03:50:49.769035 4117 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2fc28baac5404615b5abc35eb19db6ef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:50:49.769083 4117 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2fc28baac5404615b5abc35eb19db6ef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2fc28baac5404615b5abc35eb19db6ef, State: Initialized, Role: FOLLOWER
I20260806 03:50:49.769256 4117 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2fc28baac5404615b5abc35eb19db6ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2fc28baac5404615b5abc35eb19db6ef" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 42665 } }
I20260806 03:50:49.769349 4117 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2fc28baac5404615b5abc35eb19db6ef [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 03:50:49.769381 4117 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2fc28baac5404615b5abc35eb19db6ef [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 03:50:49.769434 4117 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2fc28baac5404615b5abc35eb19db6ef [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:50:49.770133 4117 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2fc28baac5404615b5abc35eb19db6ef [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2fc28baac5404615b5abc35eb19db6ef" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 42665 } }
I20260806 03:50:49.770264 4117 leader_election.cc:302] T 00000000000000000000000000000000 P 2fc28baac5404615b5abc35eb19db6ef [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2fc28baac5404615b5abc35eb19db6ef; no voters:
I20260806 03:50:49.770435 4117 leader_election.cc:288] T 00000000000000000000000000000000 P 2fc28baac5404615b5abc35eb19db6ef [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 03:50:49.770470 4123 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2fc28baac5404615b5abc35eb19db6ef [term 1 FOLLOWER]: Leader election won for term 1
I20260806 03:50:49.770634 4123 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2fc28baac5404615b5abc35eb19db6ef [term 1 LEADER]: Becoming Leader. State: Replica: 2fc28baac5404615b5abc35eb19db6ef, State: Running, Role: LEADER
I20260806 03:50:49.770694 4117 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2fc28baac5404615b5abc35eb19db6ef [sys.catalog]: configured and running, proceeding with master startup.
I20260806 03:50:49.770790 4123 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2fc28baac5404615b5abc35eb19db6ef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2fc28baac5404615b5abc35eb19db6ef" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 42665 } }
I20260806 03:50:49.771147 4123 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2fc28baac5404615b5abc35eb19db6ef [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2fc28baac5404615b5abc35eb19db6ef. Latest consensus state: current_term: 1 leader_uuid: "2fc28baac5404615b5abc35eb19db6ef" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2fc28baac5404615b5abc35eb19db6ef" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 42665 } } }
I20260806 03:50:49.771240 4123 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2fc28baac5404615b5abc35eb19db6ef [sys.catalog]: This master's current role is: LEADER
I20260806 03:50:49.771159 4124 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2fc28baac5404615b5abc35eb19db6ef [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2fc28baac5404615b5abc35eb19db6ef" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2fc28baac5404615b5abc35eb19db6ef" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 42665 } } }
I20260806 03:50:49.771400 4124 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2fc28baac5404615b5abc35eb19db6ef [sys.catalog]: This master's current role is: LEADER
I20260806 03:50:49.771448 4133 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 03:50:49.772032 4133 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 06 03:50:49 dist-test-slave-6ntk krb5kdc[4000](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988249, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.253.254@KRBTEST.COM
I20260806 03:50:49.778365 4133 catalog_manager.cc:1347] Generated new cluster ID: c19eebb3fa1f4583921227830afd12c7
I20260806 03:50:49.778447 4133 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 03:50:49.780725 4122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:50:49.766506 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35354 (local address 127.11.253.254:42665)
0806 03:50:49.766698 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0806 03:50:49.766708 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 03:50:49.766737 (+ 29us) server_negotiation.cc:407] Connection header received
0806 03:50:49.767250 (+ 513us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:50:49.767257 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:50:49.767306 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:50:49.767461 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:50:49.768100 (+ 639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:50:49.769003 (+ 903us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:50:49.770116 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:50:49.770751 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:50:49.771169 (+ 418us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:50:49.776373 (+ 5204us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:50:49.776378 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:50:49.776378 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:50:49.776390 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:50:49.778526 (+ 2136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:50:49.778945 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:50:49.778949 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:50:49.778949 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:50:49.779081 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:50:49.779359 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:50:49.779360 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:50:49.779360 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:50:49.779699 (+ 339us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:50:49.780282 (+ 583us) server_negotiation.cc:1035] Waiting for connection context
0806 03:50:49.780371 (+ 89us) server_negotiation.cc:299] Negotiation successful
0806 03:50:49.780517 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149,"thread_start_us":83,"threads_started":1}
I20260806 03:50:49.792479 4133 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 03:50:49.793213 4133 catalog_manager.cc:1504] Loading token signing keys...
I20260806 03:50:49.803963 4133 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2fc28baac5404615b5abc35eb19db6ef: Generated new TSK 0
I20260806 03:50:49.804234 4133 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.253.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.253.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.253.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.193.keytab.
Entry for principal kudu/127.11.253.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.193.keytab.
I20260806 03:50:49.841128 12279 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskToVWRe/build/release/bin/kudu
/tmp/dist-test-taskToVWRe/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785988004637995-12279-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785988004637995-12279-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785988004637995-12279-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785988004637995-12279-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.253.193:0
--local_ip_for_outbound_sockets=127.11.253.193
--webserver_interface=127.11.253.193
--webserver_port=0
--tserver_master_addrs=127.11.253.254:42665
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.193.keytab
--principal=kudu/127.11.253.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.253.212:45783
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785988004637995-12279-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785988004637995-12279-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785988004637995-12279-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:50:49.926316 4152 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:50:49.926520 4152 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:50:49.926543 4152 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:50:49.927644 4152 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:50:49.927680 4152 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:50:49.927707 4152 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.253.193
I20260806 03:50:49.929066 4152 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.253.212:45783
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785988004637995-12279-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785988004637995-12279-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.193.keytab
--principal=kudu/127.11.253.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.253.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785988004637995-12279-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.253.193
--webserver_port=0
--tserver_master_addrs=127.11.253.254:42665
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4152
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.253.193
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785988004637995-12279-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:50:49.929337 4152 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:50:49.929598 4152 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:50:49.935684 4161 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:50:49.935670 4159 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:50:49.935653 4158 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:50:49.935858 4152 server_base.cc:1034] running on GCE node
I20260806 03:50:49.936376 4152 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:50:49.936633 4152 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:50:49.937811 4152 hybrid_clock.cc:648] HybridClock initialized: now 1785988249937809 us; error 39 us; skew 500 ppm
Aug 06 03:50:49 dist-test-slave-6ntk krb5kdc[4000](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988249, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.253.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:50:49.940711 4152 init.cc:377] Logged in from keytab as kudu/127.11.253.193@KRBTEST.COM (short username kudu)
I20260806 03:50:49.941185 4152 webserver.cc:460] Webserver started at http://127.11.253.193:44889/ using document root <none> and password file <none>
I20260806 03:50:49.941445 4152 fs_manager.cc:359] Metadata directory not provided
I20260806 03:50:49.941519 4152 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:50:49.941717 4152 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:50:49.942929 4152 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785988004637995-12279-0/minicluster-data/ts-0/data/instance:
uuid: "093777e00e9442bda17b5d04e52469e8"
format_stamp: "Formatted at 2026-08-06 03:50:49 on dist-test-slave-6ntk"
server_key: "e9b30e2be5cf84a1d76d2d4951d6f824"
server_key_iv: "d32afe7f7da2dabf35c6f3db568cf5d5"
server_key_version: "encryptionkey@0"
I20260806 03:50:49.943332 4152 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785988004637995-12279-0/minicluster-data/ts-0/wal/instance:
uuid: "093777e00e9442bda17b5d04e52469e8"
format_stamp: "Formatted at 2026-08-06 03:50:49 on dist-test-slave-6ntk"
server_key: "e9b30e2be5cf84a1d76d2d4951d6f824"
server_key_iv: "d32afe7f7da2dabf35c6f3db568cf5d5"
server_key_version: "encryptionkey@0"
I20260806 03:50:49.944814 4152 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 03:50:49.945662 4170 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:50:49.945905 4152 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 03:50:49.946031 4152 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785988004637995-12279-0/minicluster-data/ts-0/data,/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785988004637995-12279-0/minicluster-data/ts-0/wal
uuid: "093777e00e9442bda17b5d04e52469e8"
format_stamp: "Formatted at 2026-08-06 03:50:49 on dist-test-slave-6ntk"
server_key: "e9b30e2be5cf84a1d76d2d4951d6f824"
server_key_iv: "d32afe7f7da2dabf35c6f3db568cf5d5"
server_key_version: "encryptionkey@0"
I20260806 03:50:49.946107 4152 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785988004637995-12279-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785988004637995-12279-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785988004637995-12279-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:50:49.963003 4152 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:50:49.965044 4152 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:50:49.965217 4152 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:50:49.965472 4152 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 03:50:49.965811 4152 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 03:50:49.965867 4152 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 03:50:49.965953 4152 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 03:50:49.965986 4152 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 03:50:49.973021 4152 rpc_server.cc:307] RPC server started. Bound to: 127.11.253.193:35771
I20260806 03:50:49.973048 4316 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.253.193:35771 every 8 connection(s)
I20260806 03:50:49.973446 4152 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785988004637995-12279-0/minicluster-data/ts-0/data/info.pb
I20260806 03:50:49.976822 12279 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskToVWRe/build/release/bin/kudu as pid 4152
I20260806 03:50:49.976918 12279 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785988004637995-12279-0/minicluster-data/ts-0/wal/instance
I20260806 03:50:49.977134 12279 external_mini_cluster.cc:1428] Setting key c3992401cfe5ae8bfd4707637bfcd20e
Aug 06 03:50:49 dist-test-slave-6ntk krb5kdc[4000](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988249, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.253.193@KRBTEST.COM for kudu/127.11.253.254@KRBTEST.COM
I20260806 03:50:49.983842 4319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:50:49.974208 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:42665 (local address 127.11.253.193:43991)
0806 03:50:49.974388 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0806 03:50:49.974431 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 03:50:49.974930 (+ 499us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:50:49.975136 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:50:49.975139 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:50:49.975753 (+ 614us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:50:49.975964 (+ 211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:50:49.975969 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:50:49.976663 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:50:49.976664 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:50:49.977715 (+ 1051us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:50:49.977717 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:50:49.978194 (+ 477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:50:49.978196 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:50:49.978331 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:50:49.978885 (+ 554us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:50:49.978894 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:50:49.980595 (+ 1701us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:50:49.982299 (+ 1704us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:50:49.982308 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:50:49.982310 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:50:49.982640 (+ 330us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:50:49.982850 (+ 210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:50:49.982851 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:50:49.982852 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:50:49.983001 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:50:49.983245 (+ 244us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:50:49.983252 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:50:49.983444 (+ 192us) client_negotiation.cc:769] Sending connection context
0806 03:50:49.983546 (+ 102us) client_negotiation.cc:240] Negotiation successful
0806 03:50:49.983671 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":106,"threads_started":1}
I20260806 03:50:49.983867 4122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:50:49.974308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.253.193:43991 (local address 127.11.253.254:42665)
0806 03:50:49.974371 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0806 03:50:49.974373 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:50:49.974509 (+ 136us) server_negotiation.cc:407] Connection header received
0806 03:50:49.975015 (+ 506us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:50:49.975016 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:50:49.975025 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:50:49.975085 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:50:49.976048 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:50:49.976578 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:50:49.977807 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:50:49.978091 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:50:49.978182 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:50:49.980703 (+ 2521us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:50:49.980704 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:50:49.980705 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:50:49.980716 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:50:49.982199 (+ 1483us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:50:49.982726 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:50:49.982726 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:50:49.982727 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:50:49.982780 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:50:49.983071 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:50:49.983071 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:50:49.983072 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:50:49.983175 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:50:49.983229 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0806 03:50:49.983605 (+ 376us) server_negotiation.cc:299] Negotiation successful
0806 03:50:49.983711 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260806 03:50:49.984367 4317 heartbeater.cc:347] Connected to a master server at 127.11.253.254:42665
I20260806 03:50:49.984490 4317 heartbeater.cc:464] Registering TS with master...
I20260806 03:50:49.984675 4317 heartbeater.cc:511] Master 127.11.253.254:42665 requested a full tablet report, sending...
I20260806 03:50:49.985252 4065 ts_manager.cc:194] Registered new tserver with Master: 093777e00e9442bda17b5d04e52469e8 (127.11.253.193:35771)
I20260806 03:50:49.986022 4065 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.253.193@KRBTEST.COM'} at 127.11.253.193:43991
WARNING: no policy specified for kudu/127.11.253.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.253.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.253.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.194.keytab.
Entry for principal kudu/127.11.253.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.194.keytab.
I20260806 03:50:50.009189 12279 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskToVWRe/build/release/bin/kudu
/tmp/dist-test-taskToVWRe/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785988004637995-12279-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785988004637995-12279-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785988004637995-12279-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785988004637995-12279-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.253.194:0
--local_ip_for_outbound_sockets=127.11.253.194
--webserver_interface=127.11.253.194
--webserver_port=0
--tserver_master_addrs=127.11.253.254:42665
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.194.keytab
--principal=kudu/127.11.253.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.253.212:45783
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785988004637995-12279-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785988004637995-12279-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785988004637995-12279-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:50:50.095088 4325 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:50:50.095348 4325 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:50:50.095386 4325 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:50:50.096534 4325 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:50:50.096607 4325 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:50:50.096652 4325 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.253.194
I20260806 03:50:50.098249 4325 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.253.212:45783
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785988004637995-12279-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785988004637995-12279-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.194.keytab
--principal=kudu/127.11.253.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.253.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785988004637995-12279-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.253.194
--webserver_port=0
--tserver_master_addrs=127.11.253.254:42665
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4325
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.253.194
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785988004637995-12279-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:50:50.098531 4325 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:50:50.098791 4325 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:50:50.104409 4337 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:50:50.104547 4334 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:50:50.104588 4325 server_base.cc:1034] running on GCE node
W20260806 03:50:50.104557 4333 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:50:50.105093 4325 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:50:50.105432 4325 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:50:50.106622 4325 hybrid_clock.cc:648] HybridClock initialized: now 1785988250106593 us; error 39 us; skew 500 ppm
Aug 06 03:50:50 dist-test-slave-6ntk krb5kdc[4000](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988250, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.253.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:50:50.109407 4325 init.cc:377] Logged in from keytab as kudu/127.11.253.194@KRBTEST.COM (short username kudu)
I20260806 03:50:50.109850 4325 webserver.cc:460] Webserver started at http://127.11.253.194:33263/ using document root <none> and password file <none>
I20260806 03:50:50.110162 4325 fs_manager.cc:359] Metadata directory not provided
I20260806 03:50:50.110213 4325 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:50:50.110309 4325 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:50:50.111413 4325 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785988004637995-12279-0/minicluster-data/ts-1/data/instance:
uuid: "6681d68f9d3c4ae490bfc176f90eb768"
format_stamp: "Formatted at 2026-08-06 03:50:50 on dist-test-slave-6ntk"
server_key: "301b182346e4dc0bca6ce4d101acd6d7"
server_key_iv: "b9925c51b99c2c15960d565a53e11f20"
server_key_version: "encryptionkey@0"
I20260806 03:50:50.111799 4325 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785988004637995-12279-0/minicluster-data/ts-1/wal/instance:
uuid: "6681d68f9d3c4ae490bfc176f90eb768"
format_stamp: "Formatted at 2026-08-06 03:50:50 on dist-test-slave-6ntk"
server_key: "301b182346e4dc0bca6ce4d101acd6d7"
server_key_iv: "b9925c51b99c2c15960d565a53e11f20"
server_key_version: "encryptionkey@0"
I20260806 03:50:50.113392 4325 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 03:50:50.114355 4346 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:50:50.114709 4325 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260806 03:50:50.114847 4325 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785988004637995-12279-0/minicluster-data/ts-1/data,/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785988004637995-12279-0/minicluster-data/ts-1/wal
uuid: "6681d68f9d3c4ae490bfc176f90eb768"
format_stamp: "Formatted at 2026-08-06 03:50:50 on dist-test-slave-6ntk"
server_key: "301b182346e4dc0bca6ce4d101acd6d7"
server_key_iv: "b9925c51b99c2c15960d565a53e11f20"
server_key_version: "encryptionkey@0"
I20260806 03:50:50.114933 4325 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785988004637995-12279-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785988004637995-12279-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785988004637995-12279-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:50:50.133270 4325 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:50:50.135892 4325 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:50:50.136075 4325 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:50:50.136353 4325 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 03:50:50.136698 4325 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 03:50:50.136735 4325 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 03:50:50.136761 4325 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 03:50:50.136806 4325 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 03:50:50.142958 4325 rpc_server.cc:307] RPC server started. Bound to: 127.11.253.194:35565
I20260806 03:50:50.143040 4497 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.253.194:35565 every 8 connection(s)
I20260806 03:50:50.143405 4325 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785988004637995-12279-0/minicluster-data/ts-1/data/info.pb
I20260806 03:50:50.145799 12279 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskToVWRe/build/release/bin/kudu as pid 4325
I20260806 03:50:50.145948 12279 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785988004637995-12279-0/minicluster-data/ts-1/wal/instance
I20260806 03:50:50.146166 12279 external_mini_cluster.cc:1428] Setting key 1a3132096ccef621e046cefb2b86fcfd
Aug 06 03:50:50 dist-test-slave-6ntk krb5kdc[4000](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988250, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.253.194@KRBTEST.COM for kudu/127.11.253.254@KRBTEST.COM
I20260806 03:50:50.154384 4122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:50:50.144291 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.253.194:52009 (local address 127.11.253.254:42665)
0806 03:50:50.144366 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0806 03:50:50.144369 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 03:50:50.144495 (+ 126us) server_negotiation.cc:407] Connection header received
0806 03:50:50.145438 (+ 943us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:50:50.145439 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:50:50.145449 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:50:50.145513 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:50:50.146583 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:50:50.147273 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:50:50.148388 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:50:50.148708 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:50:50.148789 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:50:50.151339 (+ 2550us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:50:50.151340 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:50:50.151341 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:50:50.151352 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:50:50.152858 (+ 1506us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:50:50.153341 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:50:50.153342 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:50:50.153343 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:50:50.153398 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:50:50.153661 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:50:50.153662 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:50:50.153663 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:50:50.153781 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:50:50.153857 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0806 03:50:50.154181 (+ 324us) server_negotiation.cc:299] Negotiation successful
0806 03:50:50.154280 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260806 03:50:50.154578 4501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:50:50.144169 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:42665 (local address 127.11.253.194:52009)
0806 03:50:50.144362 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0806 03:50:50.144405 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 03:50:50.145338 (+ 933us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:50:50.145626 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:50:50.145630 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:50:50.146089 (+ 459us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:50:50.146471 (+ 382us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:50:50.146477 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:50:50.147422 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:50:50.147423 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:50:50.148299 (+ 876us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:50:50.148301 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:50:50.148820 (+ 519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:50:50.148822 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:50:50.149018 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:50:50.149556 (+ 538us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:50:50.149566 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:50:50.151218 (+ 1652us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:50:50.152948 (+ 1730us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:50:50.152955 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:50:50.152958 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:50:50.153206 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:50:50.153477 (+ 271us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:50:50.153478 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:50:50.153478 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:50:50.153593 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:50:50.153859 (+ 266us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:50:50.153893 (+ 34us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:50:50.154038 (+ 145us) client_negotiation.cc:769] Sending connection context
0806 03:50:50.154154 (+ 116us) client_negotiation.cc:240] Negotiation successful
0806 03:50:50.154324 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":122,"threads_started":1}
I20260806 03:50:50.155324 4499 heartbeater.cc:347] Connected to a master server at 127.11.253.254:42665
I20260806 03:50:50.155443 4499 heartbeater.cc:464] Registering TS with master...
I20260806 03:50:50.155630 4499 heartbeater.cc:511] Master 127.11.253.254:42665 requested a full tablet report, sending...
I20260806 03:50:50.156059 4065 ts_manager.cc:194] Registered new tserver with Master: 6681d68f9d3c4ae490bfc176f90eb768 (127.11.253.194:35565)
WARNING: no policy specified for kudu/127.11.253.195@KRBTEST.COM; defaulting to no policy
I20260806 03:50:50.156608 4065 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.253.194@KRBTEST.COM'} at 127.11.253.194:52009
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.253.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.253.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.195.keytab.
Entry for principal kudu/127.11.253.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.195.keytab.
I20260806 03:50:50.177479 12279 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskToVWRe/build/release/bin/kudu
/tmp/dist-test-taskToVWRe/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785988004637995-12279-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785988004637995-12279-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785988004637995-12279-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785988004637995-12279-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.253.195:0
--local_ip_for_outbound_sockets=127.11.253.195
--webserver_interface=127.11.253.195
--webserver_port=0
--tserver_master_addrs=127.11.253.254:42665
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.195.keytab
--principal=kudu/127.11.253.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.253.212:45783
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785988004637995-12279-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785988004637995-12279-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785988004637995-12279-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:50:50.266249 4506 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:50:50.266472 4506 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:50:50.266496 4506 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:50:50.267581 4506 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:50:50.267614 4506 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:50:50.267640 4506 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.253.195
I20260806 03:50:50.269040 4506 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.253.212:45783
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785988004637995-12279-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785988004637995-12279-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.195.keytab
--principal=kudu/127.11.253.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.253.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785988004637995-12279-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.253.195
--webserver_port=0
--tserver_master_addrs=127.11.253.254:42665
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4506
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.253.195
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785988004637995-12279-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:50:50.269232 4506 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:50:50.269428 4506 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:50:50.274916 4515 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:50:50.274899 4518 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:50:50.275100 4516 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:50:50.275290 4506 server_base.cc:1034] running on GCE node
I20260806 03:50:50.275506 4506 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:50:50.275728 4506 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:50:50.276937 4506 hybrid_clock.cc:648] HybridClock initialized: now 1785988250276922 us; error 27 us; skew 500 ppm
Aug 06 03:50:50 dist-test-slave-6ntk krb5kdc[4000](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988250, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.253.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:50:50.279896 4506 init.cc:377] Logged in from keytab as kudu/127.11.253.195@KRBTEST.COM (short username kudu)
I20260806 03:50:50.280488 4506 webserver.cc:460] Webserver started at http://127.11.253.195:35223/ using document root <none> and password file <none>
I20260806 03:50:50.280771 4506 fs_manager.cc:359] Metadata directory not provided
I20260806 03:50:50.280850 4506 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:50:50.281024 4506 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:50:50.282282 4506 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785988004637995-12279-0/minicluster-data/ts-2/data/instance:
uuid: "7e7c84466bda477dadcbf8c3cfef489d"
format_stamp: "Formatted at 2026-08-06 03:50:50 on dist-test-slave-6ntk"
server_key: "84019fd53de3ed02569b2e15355db92e"
server_key_iv: "d8fa316c7767583a29474231b714feb2"
server_key_version: "encryptionkey@0"
I20260806 03:50:50.282683 4506 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785988004637995-12279-0/minicluster-data/ts-2/wal/instance:
uuid: "7e7c84466bda477dadcbf8c3cfef489d"
format_stamp: "Formatted at 2026-08-06 03:50:50 on dist-test-slave-6ntk"
server_key: "84019fd53de3ed02569b2e15355db92e"
server_key_iv: "d8fa316c7767583a29474231b714feb2"
server_key_version: "encryptionkey@0"
I20260806 03:50:50.284154 4506 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 03:50:50.285032 4528 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:50:50.285218 4506 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 03:50:50.285377 4506 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785988004637995-12279-0/minicluster-data/ts-2/data,/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785988004637995-12279-0/minicluster-data/ts-2/wal
uuid: "7e7c84466bda477dadcbf8c3cfef489d"
format_stamp: "Formatted at 2026-08-06 03:50:50 on dist-test-slave-6ntk"
server_key: "84019fd53de3ed02569b2e15355db92e"
server_key_iv: "d8fa316c7767583a29474231b714feb2"
server_key_version: "encryptionkey@0"
I20260806 03:50:50.285466 4506 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785988004637995-12279-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785988004637995-12279-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785988004637995-12279-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:50:50.293618 4506 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:50:50.296226 4506 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:50:50.296418 4506 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:50:50.296695 4506 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 03:50:50.297053 4506 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 03:50:50.297114 4506 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 03:50:50.297158 4506 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 03:50:50.297205 4506 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 03:50:50.304471 4506 rpc_server.cc:307] RPC server started. Bound to: 127.11.253.195:46617
I20260806 03:50:50.304514 4683 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.253.195:46617 every 8 connection(s)
I20260806 03:50:50.304945 4506 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785988004637995-12279-0/minicluster-data/ts-2/data/info.pb
Aug 06 03:50:50 dist-test-slave-6ntk krb5kdc[4000](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988250, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.253.195@KRBTEST.COM for kudu/127.11.253.254@KRBTEST.COM
I20260806 03:50:50.313923 12279 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskToVWRe/build/release/bin/kudu as pid 4506
I20260806 03:50:50.314030 12279 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785988004637995-12279-0/minicluster-data/ts-2/wal/instance
I20260806 03:50:50.314281 12279 external_mini_cluster.cc:1428] Setting key ae2bb5ff17c9c7287cb1043f1f779304
I20260806 03:50:50.316329 4122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:50:50.305860 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.253.195:50689 (local address 127.11.253.254:42665)
0806 03:50:50.306019 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0806 03:50:50.306022 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 03:50:50.306206 (+ 184us) server_negotiation.cc:407] Connection header received
0806 03:50:50.306839 (+ 633us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:50:50.306840 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:50:50.306856 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:50:50.306942 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:50:50.308094 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:50:50.308694 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:50:50.309893 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:50:50.310190 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:50:50.310300 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:50:50.312669 (+ 2369us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:50:50.312670 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:50:50.312671 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:50:50.312691 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:50:50.314592 (+ 1901us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:50:50.315242 (+ 650us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:50:50.315243 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:50:50.315244 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:50:50.315318 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:50:50.315654 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:50:50.315654 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:50:50.315655 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:50:50.315789 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:50:50.315844 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0806 03:50:50.316177 (+ 333us) server_negotiation.cc:299] Negotiation successful
0806 03:50:50.316256 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121}
I20260806 03:50:50.316497 4687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:50:50.305779 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:42665 (local address 127.11.253.195:50689)
0806 03:50:50.306032 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0806 03:50:50.306085 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0806 03:50:50.306710 (+ 625us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:50:50.307018 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:50:50.307023 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:50:50.307742 (+ 719us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:50:50.307959 (+ 217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:50:50.307965 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:50:50.308815 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:50:50.308816 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:50:50.309744 (+ 928us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:50:50.309746 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:50:50.310327 (+ 581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:50:50.310328 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:50:50.310475 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:50:50.311041 (+ 566us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:50:50.311049 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:50:50.312550 (+ 1501us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:50:50.314712 (+ 2162us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:50:50.314720 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:50:50.314722 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:50:50.315144 (+ 422us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:50:50.315396 (+ 252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:50:50.315397 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:50:50.315397 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:50:50.315584 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:50:50.315875 (+ 291us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:50:50.315882 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:50:50.316012 (+ 130us) client_negotiation.cc:769] Sending connection context
0806 03:50:50.316117 (+ 105us) client_negotiation.cc:240] Negotiation successful
0806 03:50:50.316259 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":119,"threads_started":1}
I20260806 03:50:50.317031 4684 heartbeater.cc:347] Connected to a master server at 127.11.253.254:42665
I20260806 03:50:50.317178 4684 heartbeater.cc:464] Registering TS with master...
I20260806 03:50:50.317400 4684 heartbeater.cc:511] Master 127.11.253.254:42665 requested a full tablet report, sending...
I20260806 03:50:50.317922 4065 ts_manager.cc:194] Registered new tserver with Master: 7e7c84466bda477dadcbf8c3cfef489d (127.11.253.195:46617)
I20260806 03:50:50.318320 4065 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.253.195@KRBTEST.COM'} at 127.11.253.195:50689
I20260806 03:50:50.326536 12279 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 03:50:50.334228 4122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:50:50.327870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35360 (local address 127.11.253.254:42665)
0806 03:50:50.327913 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0806 03:50:50.327915 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:50:50.328027 (+ 112us) server_negotiation.cc:407] Connection header received
0806 03:50:50.328079 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:50:50.328079 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:50:50.328089 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:50:50.328140 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:50:50.328716 (+ 576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:50:50.329183 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:50:50.330068 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:50:50.330279 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:50:50.330345 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:50:50.331626 (+ 1281us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:50:50.331628 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:50:50.331628 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:50:50.331635 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:50:50.333109 (+ 1474us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:50:50.333457 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:50:50.333458 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:50:50.333458 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:50:50.333517 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:50:50.333694 (+ 177us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:50:50.333695 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:50:50.333695 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:50:50.333841 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:50:50.334004 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0806 03:50:50.334082 (+ 78us) server_negotiation.cc:299] Negotiation successful
0806 03:50:50.334146 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260806 03:50:50.335940 4065 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35360:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 03:50:50.337617 4065 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 03:50:50.344662 4701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:50:50.340298 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.193:35771 (local address 127.0.0.1:45718)
0806 03:50:50.340501 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0806 03:50:50.340512 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 03:50:50.340569 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:50:50.341149 (+ 580us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:50:50.341151 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:50:50.341153 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 03:50:50.341199 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:50:50.341200 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:50:50.342544 (+ 1344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:50:50.342545 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:50:50.343607 (+ 1062us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:50:50.343609 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:50:50.344421 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:50:50.344421 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:50:50.344478 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:50:50.344481 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 03:50:50.344542 (+ 61us) client_negotiation.cc:240] Negotiation successful
0806 03:50:50.344574 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":152,"threads_started":1}
I20260806 03:50:50.344760 4704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:50:50.340673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55312 (local address 127.11.253.195:46617)
0806 03:50:50.341017 (+ 344us) server_negotiation.cc:206] Beginning negotiation
0806 03:50:50.341021 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 03:50:50.341031 (+ 10us) server_negotiation.cc:407] Connection header received
0806 03:50:50.341069 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:50:50.341071 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:50:50.341085 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:50:50.341162 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 03:50:50.341398 (+ 236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:50:50.342492 (+ 1094us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:50:50.343607 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:50:50.344134 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:50:50.344232 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:50:50.344305 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0806 03:50:50.344620 (+ 315us) server_negotiation.cc:299] Negotiation successful
0806 03:50:50.344666 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":310,"thread_start_us":128,"threads_started":1}
I20260806 03:50:50.344884 4700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:50:50.340209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53172 (local address 127.11.253.194:35565)
0806 03:50:50.340668 (+ 459us) server_negotiation.cc:206] Beginning negotiation
0806 03:50:50.340675 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 03:50:50.340688 (+ 13us) server_negotiation.cc:407] Connection header received
0806 03:50:50.340734 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:50:50.340737 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:50:50.340753 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:50:50.340857 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 03:50:50.341097 (+ 240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:50:50.342070 (+ 973us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:50:50.343557 (+ 1487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:50:50.344287 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:50:50.344370 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:50:50.344432 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0806 03:50:50.344705 (+ 273us) server_negotiation.cc:299] Negotiation successful
0806 03:50:50.344782 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":417,"thread_start_us":124,"threads_started":1}
I20260806 03:50:50.344920 4697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:50:50.340058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.194:35565 (local address 127.0.0.1:53172)
0806 03:50:50.340184 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0806 03:50:50.340226 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 03:50:50.340290 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:50:50.340861 (+ 571us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:50:50.340865 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:50:50.340867 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 03:50:50.341012 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:50:50.341015 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:50:50.342190 (+ 1175us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:50:50.342191 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:50:50.343458 (+ 1267us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:50:50.343461 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:50:50.344473 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:50:50.344474 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:50:50.344546 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:50:50.344551 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 03:50:50.344602 (+ 51us) client_negotiation.cc:240] Negotiation successful
0806 03:50:50.344644 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":98,"mutex_wait_us":1,"thread_start_us":77,"threads_started":1}
I20260806 03:50:50.344574 4703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:50:50.340568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.195:46617 (local address 127.0.0.1:55312)
0806 03:50:50.340677 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0806 03:50:50.340687 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 03:50:50.340727 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:50:50.341293 (+ 566us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:50:50.341294 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:50:50.341296 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 03:50:50.341325 (+ 29us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:50:50.341326 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:50:50.342580 (+ 1254us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:50:50.342581 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:50:50.343491 (+ 910us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:50:50.343493 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:50:50.344229 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:50:50.344230 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:50:50.344309 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:50:50.344323 (+ 14us) client_negotiation.cc:769] Sending connection context
0806 03:50:50.344421 (+ 98us) client_negotiation.cc:240] Negotiation successful
0806 03:50:50.344478 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":92,"thread_start_us":76,"threads_started":1}
I20260806 03:50:50.345227 4702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:50:50.340440 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45718 (local address 127.11.253.193:35771)
0806 03:50:50.340952 (+ 512us) server_negotiation.cc:206] Beginning negotiation
0806 03:50:50.340957 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 03:50:50.340968 (+ 11us) server_negotiation.cc:407] Connection header received
0806 03:50:50.341011 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:50:50.341013 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:50:50.341031 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:50:50.341150 (+ 119us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 03:50:50.341278 (+ 128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:50:50.342434 (+ 1156us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:50:50.343671 (+ 1237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:50:50.344338 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:50:50.344404 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:50:50.344450 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0806 03:50:50.344558 (+ 108us) server_negotiation.cc:299] Negotiation successful
0806 03:50:50.344612 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":466,"thread_start_us":90,"threads_started":1}
I20260806 03:50:50.345908 4232 tablet_service.cc:1467] Processing CreateTablet for tablet 1320c13c35bb48098e931db21b5b1bef (DEFAULT_TABLE table=test-table [id=1139998813624c95b071ff565dad510f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 03:50:50.345949 4413 tablet_service.cc:1467] Processing CreateTablet for tablet 1320c13c35bb48098e931db21b5b1bef (DEFAULT_TABLE table=test-table [id=1139998813624c95b071ff565dad510f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 03:50:50.345963 4594 tablet_service.cc:1467] Processing CreateTablet for tablet 1320c13c35bb48098e931db21b5b1bef (DEFAULT_TABLE table=test-table [id=1139998813624c95b071ff565dad510f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 03:50:50.346320 4232 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1320c13c35bb48098e931db21b5b1bef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:50:50.346323 4594 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1320c13c35bb48098e931db21b5b1bef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:50:50.346323 4413 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1320c13c35bb48098e931db21b5b1bef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:50:50.349213 4705 tablet_bootstrap.cc:492] T 1320c13c35bb48098e931db21b5b1bef P 093777e00e9442bda17b5d04e52469e8: Bootstrap starting.
I20260806 03:50:50.349678 4710 tablet_bootstrap.cc:492] T 1320c13c35bb48098e931db21b5b1bef P 6681d68f9d3c4ae490bfc176f90eb768: Bootstrap starting.
I20260806 03:50:50.350021 4708 tablet_bootstrap.cc:492] T 1320c13c35bb48098e931db21b5b1bef P 7e7c84466bda477dadcbf8c3cfef489d: Bootstrap starting.
I20260806 03:50:50.350217 4705 tablet_bootstrap.cc:654] T 1320c13c35bb48098e931db21b5b1bef P 093777e00e9442bda17b5d04e52469e8: Neither blocks nor log segments found. Creating new log.
I20260806 03:50:50.350376 4710 tablet_bootstrap.cc:654] T 1320c13c35bb48098e931db21b5b1bef P 6681d68f9d3c4ae490bfc176f90eb768: Neither blocks nor log segments found. Creating new log.
I20260806 03:50:50.350683 4708 tablet_bootstrap.cc:654] T 1320c13c35bb48098e931db21b5b1bef P 7e7c84466bda477dadcbf8c3cfef489d: Neither blocks nor log segments found. Creating new log.
I20260806 03:50:50.350871 4705 log.cc:824] T 1320c13c35bb48098e931db21b5b1bef P 093777e00e9442bda17b5d04e52469e8: Log is configured to *not* fsync() on all Append() calls
I20260806 03:50:50.351001 4710 log.cc:824] T 1320c13c35bb48098e931db21b5b1bef P 6681d68f9d3c4ae490bfc176f90eb768: Log is configured to *not* fsync() on all Append() calls
I20260806 03:50:50.351290 4708 log.cc:824] T 1320c13c35bb48098e931db21b5b1bef P 7e7c84466bda477dadcbf8c3cfef489d: Log is configured to *not* fsync() on all Append() calls
I20260806 03:50:50.351706 4705 tablet_bootstrap.cc:492] T 1320c13c35bb48098e931db21b5b1bef P 093777e00e9442bda17b5d04e52469e8: No bootstrap required, opened a new log
I20260806 03:50:50.351773 4705 ts_tablet_manager.cc:1397] T 1320c13c35bb48098e931db21b5b1bef P 093777e00e9442bda17b5d04e52469e8: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 03:50:50.352035 4708 tablet_bootstrap.cc:492] T 1320c13c35bb48098e931db21b5b1bef P 7e7c84466bda477dadcbf8c3cfef489d: No bootstrap required, opened a new log
I20260806 03:50:50.352124 4708 ts_tablet_manager.cc:1397] T 1320c13c35bb48098e931db21b5b1bef P 7e7c84466bda477dadcbf8c3cfef489d: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 03:50:50.353188 4705 raft_consensus.cc:348] T 1320c13c35bb48098e931db21b5b1bef P 093777e00e9442bda17b5d04e52469e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6681d68f9d3c4ae490bfc176f90eb768" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 35565 } } peers { permanent_uuid: "093777e00e9442bda17b5d04e52469e8" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 35771 } } peers { permanent_uuid: "7e7c84466bda477dadcbf8c3cfef489d" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 46617 } }
I20260806 03:50:50.353338 4705 raft_consensus.cc:374] T 1320c13c35bb48098e931db21b5b1bef P 093777e00e9442bda17b5d04e52469e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:50:50.353363 4705 raft_consensus.cc:729] T 1320c13c35bb48098e931db21b5b1bef P 093777e00e9442bda17b5d04e52469e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 093777e00e9442bda17b5d04e52469e8, State: Initialized, Role: FOLLOWER
I20260806 03:50:50.353489 4705 consensus_queue.cc:260] T 1320c13c35bb48098e931db21b5b1bef P 093777e00e9442bda17b5d04e52469e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6681d68f9d3c4ae490bfc176f90eb768" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 35565 } } peers { permanent_uuid: "093777e00e9442bda17b5d04e52469e8" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 35771 } } peers { permanent_uuid: "7e7c84466bda477dadcbf8c3cfef489d" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 46617 } }
I20260806 03:50:50.353614 4708 raft_consensus.cc:348] T 1320c13c35bb48098e931db21b5b1bef P 7e7c84466bda477dadcbf8c3cfef489d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6681d68f9d3c4ae490bfc176f90eb768" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 35565 } } peers { permanent_uuid: "093777e00e9442bda17b5d04e52469e8" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 35771 } } peers { permanent_uuid: "7e7c84466bda477dadcbf8c3cfef489d" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 46617 } }
I20260806 03:50:50.353802 4708 raft_consensus.cc:374] T 1320c13c35bb48098e931db21b5b1bef P 7e7c84466bda477dadcbf8c3cfef489d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:50:50.353807 4705 ts_tablet_manager.cc:1428] T 1320c13c35bb48098e931db21b5b1bef P 093777e00e9442bda17b5d04e52469e8: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 03:50:50.353830 4317 heartbeater.cc:503] Master 127.11.253.254:42665 was elected leader, sending a full tablet report...
I20260806 03:50:50.353857 4708 raft_consensus.cc:729] T 1320c13c35bb48098e931db21b5b1bef P 7e7c84466bda477dadcbf8c3cfef489d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e7c84466bda477dadcbf8c3cfef489d, State: Initialized, Role: FOLLOWER
I20260806 03:50:50.354003 4708 consensus_queue.cc:260] T 1320c13c35bb48098e931db21b5b1bef P 7e7c84466bda477dadcbf8c3cfef489d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6681d68f9d3c4ae490bfc176f90eb768" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 35565 } } peers { permanent_uuid: "093777e00e9442bda17b5d04e52469e8" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 35771 } } peers { permanent_uuid: "7e7c84466bda477dadcbf8c3cfef489d" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 46617 } }
I20260806 03:50:50.354277 4710 tablet_bootstrap.cc:492] T 1320c13c35bb48098e931db21b5b1bef P 6681d68f9d3c4ae490bfc176f90eb768: No bootstrap required, opened a new log
I20260806 03:50:50.354326 4708 ts_tablet_manager.cc:1428] T 1320c13c35bb48098e931db21b5b1bef P 7e7c84466bda477dadcbf8c3cfef489d: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260806 03:50:50.354362 4710 ts_tablet_manager.cc:1397] T 1320c13c35bb48098e931db21b5b1bef P 6681d68f9d3c4ae490bfc176f90eb768: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260806 03:50:50.354471 4684 heartbeater.cc:503] Master 127.11.253.254:42665 was elected leader, sending a full tablet report...
I20260806 03:50:50.356295 4710 raft_consensus.cc:348] T 1320c13c35bb48098e931db21b5b1bef P 6681d68f9d3c4ae490bfc176f90eb768 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6681d68f9d3c4ae490bfc176f90eb768" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 35565 } } peers { permanent_uuid: "093777e00e9442bda17b5d04e52469e8" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 35771 } } peers { permanent_uuid: "7e7c84466bda477dadcbf8c3cfef489d" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 46617 } }
I20260806 03:50:50.356441 4710 raft_consensus.cc:374] T 1320c13c35bb48098e931db21b5b1bef P 6681d68f9d3c4ae490bfc176f90eb768 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:50:50.356462 4710 raft_consensus.cc:729] T 1320c13c35bb48098e931db21b5b1bef P 6681d68f9d3c4ae490bfc176f90eb768 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6681d68f9d3c4ae490bfc176f90eb768, State: Initialized, Role: FOLLOWER
I20260806 03:50:50.356570 4710 consensus_queue.cc:260] T 1320c13c35bb48098e931db21b5b1bef P 6681d68f9d3c4ae490bfc176f90eb768 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6681d68f9d3c4ae490bfc176f90eb768" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 35565 } } peers { permanent_uuid: "093777e00e9442bda17b5d04e52469e8" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 35771 } } peers { permanent_uuid: "7e7c84466bda477dadcbf8c3cfef489d" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 46617 } }
I20260806 03:50:50.356865 4710 ts_tablet_manager.cc:1428] T 1320c13c35bb48098e931db21b5b1bef P 6681d68f9d3c4ae490bfc176f90eb768: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260806 03:50:50.356873 4499 heartbeater.cc:503] Master 127.11.253.254:42665 was elected leader, sending a full tablet report...
W20260806 03:50:50.394109 4500 tablet.cc:2369] T 1320c13c35bb48098e931db21b5b1bef P 6681d68f9d3c4ae490bfc176f90eb768: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 03:50:50.399919 4720 raft_consensus.cc:482] T 1320c13c35bb48098e931db21b5b1bef P 6681d68f9d3c4ae490bfc176f90eb768 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 03:50:50.400035 4720 raft_consensus.cc:504] T 1320c13c35bb48098e931db21b5b1bef P 6681d68f9d3c4ae490bfc176f90eb768 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6681d68f9d3c4ae490bfc176f90eb768" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 35565 } } peers { permanent_uuid: "093777e00e9442bda17b5d04e52469e8" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 35771 } } peers { permanent_uuid: "7e7c84466bda477dadcbf8c3cfef489d" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 46617 } }
I20260806 03:50:50.400444 4720 leader_election.cc:288] T 1320c13c35bb48098e931db21b5b1bef P 6681d68f9d3c4ae490bfc176f90eb768 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 093777e00e9442bda17b5d04e52469e8 (127.11.253.193:35771), 7e7c84466bda477dadcbf8c3cfef489d (127.11.253.195:46617)
I20260806 03:50:50.403352 4702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:50:50.400707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.253.194:36567 (local address 127.11.253.193:35771)
0806 03:50:50.400777 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0806 03:50:50.400779 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:50:50.400789 (+ 10us) server_negotiation.cc:407] Connection header received
0806 03:50:50.400837 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:50:50.400838 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:50:50.400849 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:50:50.400918 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 03:50:50.401081 (+ 163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:50:50.401580 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:50:50.402600 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:50:50.402982 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:50:50.403070 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:50:50.403078 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0806 03:50:50.403251 (+ 173us) server_negotiation.cc:299] Negotiation successful
0806 03:50:50.403289 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260806 03:50:50.403352 4501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:50:50.400628 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.193:35771 (local address 127.11.253.194:36567)
0806 03:50:50.400690 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0806 03:50:50.400703 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 03:50:50.400779 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:50:50.400921 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:50:50.400922 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:50:50.400923 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 03:50:50.401012 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:50:50.401014 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:50:50.401676 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:50:50.401677 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:50:50.402506 (+ 829us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:50:50.402509 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:50:50.403064 (+ 555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:50:50.403065 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:50:50.403151 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:50:50.403153 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 03:50:50.403236 (+ 83us) client_negotiation.cc:240] Negotiation successful
0806 03:50:50.403278 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40}
I20260806 03:50:50.403585 4704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:50:50.401015 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.253.194:60047 (local address 127.11.253.195:46617)
0806 03:50:50.401089 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0806 03:50:50.401092 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 03:50:50.401100 (+ 8us) server_negotiation.cc:407] Connection header received
0806 03:50:50.401128 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:50:50.401129 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:50:50.401138 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:50:50.401207 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 03:50:50.401379 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:50:50.401846 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:50:50.402818 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:50:50.403205 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:50:50.403293 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:50:50.403302 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0806 03:50:50.403489 (+ 187us) server_negotiation.cc:299] Negotiation successful
0806 03:50:50.403540 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260806 03:50:50.403613 4721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:50:50.400769 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.195:46617 (local address 127.11.253.194:60047)
0806 03:50:50.400979 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0806 03:50:50.400990 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 03:50:50.401067 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:50:50.401226 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:50:50.401226 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:50:50.401228 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 03:50:50.401307 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:50:50.401308 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:50:50.401982 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:50:50.401983 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:50:50.402720 (+ 737us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:50:50.402722 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:50:50.403308 (+ 586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:50:50.403309 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:50:50.403381 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:50:50.403384 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 03:50:50.403456 (+ 72us) client_negotiation.cc:240] Negotiation successful
0806 03:50:50.403506 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":130,"threads_started":1}
I20260806 03:50:50.403879 4260 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1320c13c35bb48098e931db21b5b1bef" candidate_uuid: "6681d68f9d3c4ae490bfc176f90eb768" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "093777e00e9442bda17b5d04e52469e8" is_pre_election: true
I20260806 03:50:50.404043 4260 raft_consensus.cc:2454] T 1320c13c35bb48098e931db21b5b1bef P 093777e00e9442bda17b5d04e52469e8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6681d68f9d3c4ae490bfc176f90eb768 in term 0.
I20260806 03:50:50.404235 4620 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1320c13c35bb48098e931db21b5b1bef" candidate_uuid: "6681d68f9d3c4ae490bfc176f90eb768" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e7c84466bda477dadcbf8c3cfef489d" is_pre_election: true
I20260806 03:50:50.404289 4348 leader_election.cc:302] T 1320c13c35bb48098e931db21b5b1bef P 6681d68f9d3c4ae490bfc176f90eb768 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 093777e00e9442bda17b5d04e52469e8, 6681d68f9d3c4ae490bfc176f90eb768; no voters:
I20260806 03:50:50.404385 4620 raft_consensus.cc:2454] T 1320c13c35bb48098e931db21b5b1bef P 7e7c84466bda477dadcbf8c3cfef489d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6681d68f9d3c4ae490bfc176f90eb768 in term 0.
I20260806 03:50:50.404412 4720 raft_consensus.cc:2781] T 1320c13c35bb48098e931db21b5b1bef P 6681d68f9d3c4ae490bfc176f90eb768 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 03:50:50.404466 4720 raft_consensus.cc:482] T 1320c13c35bb48098e931db21b5b1bef P 6681d68f9d3c4ae490bfc176f90eb768 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 03:50:50.404498 4720 raft_consensus.cc:3037] T 1320c13c35bb48098e931db21b5b1bef P 6681d68f9d3c4ae490bfc176f90eb768 [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:50:50.405300 4720 raft_consensus.cc:504] T 1320c13c35bb48098e931db21b5b1bef P 6681d68f9d3c4ae490bfc176f90eb768 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6681d68f9d3c4ae490bfc176f90eb768" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 35565 } } peers { permanent_uuid: "093777e00e9442bda17b5d04e52469e8" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 35771 } } peers { permanent_uuid: "7e7c84466bda477dadcbf8c3cfef489d" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 46617 } }
I20260806 03:50:50.405441 4720 leader_election.cc:288] T 1320c13c35bb48098e931db21b5b1bef P 6681d68f9d3c4ae490bfc176f90eb768 [CANDIDATE]: Term 1 election: Requested vote from peers 093777e00e9442bda17b5d04e52469e8 (127.11.253.193:35771), 7e7c84466bda477dadcbf8c3cfef489d (127.11.253.195:46617)
I20260806 03:50:50.405674 4620 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1320c13c35bb48098e931db21b5b1bef" candidate_uuid: "6681d68f9d3c4ae490bfc176f90eb768" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e7c84466bda477dadcbf8c3cfef489d"
I20260806 03:50:50.405683 4260 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1320c13c35bb48098e931db21b5b1bef" candidate_uuid: "6681d68f9d3c4ae490bfc176f90eb768" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "093777e00e9442bda17b5d04e52469e8"
I20260806 03:50:50.405764 4620 raft_consensus.cc:3037] T 1320c13c35bb48098e931db21b5b1bef P 7e7c84466bda477dadcbf8c3cfef489d [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:50:50.405764 4260 raft_consensus.cc:3037] T 1320c13c35bb48098e931db21b5b1bef P 093777e00e9442bda17b5d04e52469e8 [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:50:50.406744 4620 raft_consensus.cc:2454] T 1320c13c35bb48098e931db21b5b1bef P 7e7c84466bda477dadcbf8c3cfef489d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6681d68f9d3c4ae490bfc176f90eb768 in term 1.
I20260806 03:50:50.406744 4260 raft_consensus.cc:2454] T 1320c13c35bb48098e931db21b5b1bef P 093777e00e9442bda17b5d04e52469e8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6681d68f9d3c4ae490bfc176f90eb768 in term 1.
I20260806 03:50:50.406975 4348 leader_election.cc:302] T 1320c13c35bb48098e931db21b5b1bef P 6681d68f9d3c4ae490bfc176f90eb768 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 093777e00e9442bda17b5d04e52469e8, 6681d68f9d3c4ae490bfc176f90eb768; no voters:
I20260806 03:50:50.407079 4720 raft_consensus.cc:2781] T 1320c13c35bb48098e931db21b5b1bef P 6681d68f9d3c4ae490bfc176f90eb768 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 03:50:50.407243 4720 raft_consensus.cc:686] T 1320c13c35bb48098e931db21b5b1bef P 6681d68f9d3c4ae490bfc176f90eb768 [term 1 LEADER]: Becoming Leader. State: Replica: 6681d68f9d3c4ae490bfc176f90eb768, State: Running, Role: LEADER
I20260806 03:50:50.407341 4720 consensus_queue.cc:237] T 1320c13c35bb48098e931db21b5b1bef P 6681d68f9d3c4ae490bfc176f90eb768 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6681d68f9d3c4ae490bfc176f90eb768" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 35565 } } peers { permanent_uuid: "093777e00e9442bda17b5d04e52469e8" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 35771 } } peers { permanent_uuid: "7e7c84466bda477dadcbf8c3cfef489d" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 46617 } }
I20260806 03:50:50.408180 4065 catalog_manager.cc:5495] T 1320c13c35bb48098e931db21b5b1bef P 6681d68f9d3c4ae490bfc176f90eb768 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6681d68f9d3c4ae490bfc176f90eb768 (127.11.253.194). New cstate: current_term: 1 leader_uuid: "6681d68f9d3c4ae490bfc176f90eb768" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6681d68f9d3c4ae490bfc176f90eb768" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 35565 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "093777e00e9442bda17b5d04e52469e8" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 35771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e7c84466bda477dadcbf8c3cfef489d" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 46617 } health_report { overall_health: UNKNOWN } } }
I20260806 03:50:50.417519 4700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:50:50.415044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53184 (local address 127.11.253.194:35565)
0806 03:50:50.415103 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0806 03:50:50.415105 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:50:50.415114 (+ 9us) server_negotiation.cc:407] Connection header received
0806 03:50:50.415151 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:50:50.415152 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:50:50.415178 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:50:50.415320 (+ 142us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 03:50:50.415449 (+ 129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:50:50.415908 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:50:50.416670 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:50:50.416866 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:50:50.416944 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:50:50.417128 (+ 184us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 03:50:50.417261 (+ 133us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 03:50:50.417366 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0806 03:50:50.417392 (+ 26us) server_negotiation.cc:299] Negotiation successful
0806 03:50:50.417411 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260806 03:50:50.419783 4620 raft_consensus.cc:1261] T 1320c13c35bb48098e931db21b5b1bef P 7e7c84466bda477dadcbf8c3cfef489d [term 1 FOLLOWER]: Refusing update from remote peer 6681d68f9d3c4ae490bfc176f90eb768: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 03:50:50.419842 4260 raft_consensus.cc:1261] T 1320c13c35bb48098e931db21b5b1bef P 093777e00e9442bda17b5d04e52469e8 [term 1 FOLLOWER]: Refusing update from remote peer 6681d68f9d3c4ae490bfc176f90eb768: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 03:50:50.420184 4720 consensus_queue.cc:1035] T 1320c13c35bb48098e931db21b5b1bef P 6681d68f9d3c4ae490bfc176f90eb768 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e7c84466bda477dadcbf8c3cfef489d" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 46617 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 03:50:50.420323 4723 consensus_queue.cc:1035] T 1320c13c35bb48098e931db21b5b1bef P 6681d68f9d3c4ae490bfc176f90eb768 [LEADER]: Connected to new peer: Peer: permanent_uuid: "093777e00e9442bda17b5d04e52469e8" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 35771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 03:50:50.422946 4727 mvcc.cc:205] Tried to move back new op lower bound from 7315407873716314112 to 7315407873668935680. Current Snapshot: MvccSnapshot[applied={T|T < 7315407873716314112}]
I20260806 03:50:50.424871 4728 mvcc.cc:205] Tried to move back new op lower bound from 7315407873716314112 to 7315407873668935680. Current Snapshot: MvccSnapshot[applied={T|T < 7315407873716314112 or (T in {7315407873716314112})}]
I20260806 03:50:50.425189 4065 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35360:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 03:50:50.425292 4065 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35360:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 03:50:50.426081 4065 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2fc28baac5404615b5abc35eb19db6ef: Sending DeleteTablet for 3 replicas of tablet 1320c13c35bb48098e931db21b5b1bef
I20260806 03:50:50.426427 4413 tablet_service.cc:1514] Processing DeleteTablet for tablet 1320c13c35bb48098e931db21b5b1bef with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 03:50:50 UTC) from {username='kudu', principal='kudu/127.11.253.254@KRBTEST.COM'} at 127.0.0.1:53172
I20260806 03:50:50.426440 4232 tablet_service.cc:1514] Processing DeleteTablet for tablet 1320c13c35bb48098e931db21b5b1bef with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 03:50:50 UTC) from {username='kudu', principal='kudu/127.11.253.254@KRBTEST.COM'} at 127.0.0.1:45718
I20260806 03:50:50.426505 4594 tablet_service.cc:1514] Processing DeleteTablet for tablet 1320c13c35bb48098e931db21b5b1bef with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 03:50:50 UTC) from {username='kudu', principal='kudu/127.11.253.254@KRBTEST.COM'} at 127.0.0.1:55312
I20260806 03:50:50.426736 4741 tablet_replica.cc:330] T 1320c13c35bb48098e931db21b5b1bef P 7e7c84466bda477dadcbf8c3cfef489d: stopping tablet replica
I20260806 03:50:50.426843 4741 raft_consensus.cc:2229] T 1320c13c35bb48098e931db21b5b1bef P 7e7c84466bda477dadcbf8c3cfef489d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 03:50:50.426874 4740 tablet_replica.cc:330] T 1320c13c35bb48098e931db21b5b1bef P 093777e00e9442bda17b5d04e52469e8: stopping tablet replica
I20260806 03:50:50.426949 4741 raft_consensus.cc:2258] T 1320c13c35bb48098e931db21b5b1bef P 7e7c84466bda477dadcbf8c3cfef489d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 03:50:50.426965 4740 raft_consensus.cc:2229] T 1320c13c35bb48098e931db21b5b1bef P 093777e00e9442bda17b5d04e52469e8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 03:50:50.427095 12279 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskToVWRe/build/release/bin/kudu with pid 4152
I20260806 03:50:50.431746 4739 tablet_replica.cc:330] T 1320c13c35bb48098e931db21b5b1bef P 6681d68f9d3c4ae490bfc176f90eb768: stopping tablet replica
I20260806 03:50:50.431839 4739 raft_consensus.cc:2229] T 1320c13c35bb48098e931db21b5b1bef P 6681d68f9d3c4ae490bfc176f90eb768 [term 1 LEADER]: Raft consensus shutting down.
I20260806 03:50:50.431922 4739 raft_consensus.cc:2258] T 1320c13c35bb48098e931db21b5b1bef P 6681d68f9d3c4ae490bfc176f90eb768 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 03:50:50.450649 4741 ts_tablet_manager.cc:1905] T 1320c13c35bb48098e931db21b5b1bef P 7e7c84466bda477dadcbf8c3cfef489d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 03:50:50.451038 4739 ts_tablet_manager.cc:1905] T 1320c13c35bb48098e931db21b5b1bef P 6681d68f9d3c4ae490bfc176f90eb768: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 03:50:50.452824 4741 ts_tablet_manager.cc:1918] T 1320c13c35bb48098e931db21b5b1bef P 7e7c84466bda477dadcbf8c3cfef489d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 03:50:50.452927 4741 log.cc:1196] T 1320c13c35bb48098e931db21b5b1bef P 7e7c84466bda477dadcbf8c3cfef489d: Deleting WAL directory at /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785988004637995-12279-0/minicluster-data/ts-2/wal/wals/1320c13c35bb48098e931db21b5b1bef
I20260806 03:50:50.452883 4739 ts_tablet_manager.cc:1918] T 1320c13c35bb48098e931db21b5b1bef P 6681d68f9d3c4ae490bfc176f90eb768: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 03:50:50.453107 4739 log.cc:1196] T 1320c13c35bb48098e931db21b5b1bef P 6681d68f9d3c4ae490bfc176f90eb768: Deleting WAL directory at /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785988004637995-12279-0/minicluster-data/ts-1/wal/wals/1320c13c35bb48098e931db21b5b1bef
I20260806 03:50:50.453249 4741 ts_tablet_manager.cc:1939] T 1320c13c35bb48098e931db21b5b1bef P 7e7c84466bda477dadcbf8c3cfef489d: Deleting consensus metadata
I20260806 03:50:50.453480 4739 ts_tablet_manager.cc:1939] T 1320c13c35bb48098e931db21b5b1bef P 6681d68f9d3c4ae490bfc176f90eb768: Deleting consensus metadata
I20260806 03:50:50.453804 4040 catalog_manager.cc:4841] TS 7e7c84466bda477dadcbf8c3cfef489d (127.11.253.195:46617): tablet 1320c13c35bb48098e931db21b5b1bef (table test-table [id=1139998813624c95b071ff565dad510f]) successfully deleted
I20260806 03:50:50.453948 4040 catalog_manager.cc:4841] TS 6681d68f9d3c4ae490bfc176f90eb768 (127.11.253.194:35565): tablet 1320c13c35bb48098e931db21b5b1bef (table test-table [id=1139998813624c95b071ff565dad510f]) successfully deleted
W20260806 03:50:50.455374 4040 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.253.193:35771 (error 108)
I20260806 03:50:50.456015 4703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:50:50.455867 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.193:35771 (local address 127.0.0.1:45722)
0806 03:50:50.455935 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0806 03:50:50.455979 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.253.193:35771: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260806 03:50:50.456065 12279 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskToVWRe/build/release/bin/kudu with pid 4325
W20260806 03:50:50.458595 4040 catalog_manager.cc:4584] TS 093777e00e9442bda17b5d04e52469e8 (127.11.253.193:35771): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 1320c13c35bb48098e931db21b5b1bef: Network error: Client connection negotiation failed: client connection to 127.11.253.193:35771: connect: Connection refused (error 111)
I20260806 03:50:50.465952 12279 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskToVWRe/build/release/bin/kudu with pid 4506
I20260806 03:50:50.473665 12279 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskToVWRe/build/release/bin/kudu with pid 4018
2026-08-06T03:50:50Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (2224 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-06T03:50:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T03:50:50Z Disabled control of system clock
I20260806 03:50:50.507550 12279 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskToVWRe/build/release/bin/kudu
/tmp/dist-test-taskToVWRe/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785988004637995-12279-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785988004637995-12279-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785988004637995-12279-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785988004637995-12279-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.253.254:35281
--webserver_interface=127.11.253.254
--webserver_port=0
--builtin_ntp_servers=127.11.253.212:37747
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.253.254:35281
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 03:50:50.593319 4751 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:50:50.593602 4751 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:50:50.593644 4751 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:50:50.595402 4751 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 03:50:50.595448 4751 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:50:50.595461 4751 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:50:50.595474 4751 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 03:50:50.595487 4751 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 03:50:50.596794 4751 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.253.212:37747
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785988004637995-12279-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785988004637995-12279-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.253.254:35281
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.253.254:35281
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785988004637995-12279-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.253.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4751
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785988004637995-12279-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:50:50.597009 4751 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:50:50.597246 4751 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:50:50.602465 4758 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:50:50.602458 4761 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:50:50.602478 4759 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:50:50.602672 4751 server_base.cc:1034] running on GCE node
I20260806 03:50:50.602975 4751 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:50:50.603235 4751 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:50:50.604372 4751 hybrid_clock.cc:648] HybridClock initialized: now 1785988250604333 us; error 57 us; skew 500 ppm
I20260806 03:50:50.605810 4751 webserver.cc:460] Webserver started at http://127.11.253.254:43687/ using document root <none> and password file <none>
I20260806 03:50:50.606096 4751 fs_manager.cc:359] Metadata directory not provided
I20260806 03:50:50.606148 4751 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:50:50.606297 4751 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:50:50.607412 4751 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785988004637995-12279-0/minicluster-data/master-0/data/instance:
uuid: "237c89cd5c5d4a50877ab4a0c0e27903"
format_stamp: "Formatted at 2026-08-06 03:50:50 on dist-test-slave-6ntk"
server_key: "8d6e269ee6fca27662af14fde67efd22"
server_key_iv: "0aa374df3c93142dea2b4bab2381b4c0"
server_key_version: "encryptionkey@0"
I20260806 03:50:50.607725 4751 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785988004637995-12279-0/minicluster-data/master-0/wal/instance:
uuid: "237c89cd5c5d4a50877ab4a0c0e27903"
format_stamp: "Formatted at 2026-08-06 03:50:50 on dist-test-slave-6ntk"
server_key: "8d6e269ee6fca27662af14fde67efd22"
server_key_iv: "0aa374df3c93142dea2b4bab2381b4c0"
server_key_version: "encryptionkey@0"
I20260806 03:50:50.609088 4751 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 03:50:50.609951 4768 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:50:50.610152 4751 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 03:50:50.610268 4751 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785988004637995-12279-0/minicluster-data/master-0/data,/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785988004637995-12279-0/minicluster-data/master-0/wal
uuid: "237c89cd5c5d4a50877ab4a0c0e27903"
format_stamp: "Formatted at 2026-08-06 03:50:50 on dist-test-slave-6ntk"
server_key: "8d6e269ee6fca27662af14fde67efd22"
server_key_iv: "0aa374df3c93142dea2b4bab2381b4c0"
server_key_version: "encryptionkey@0"
I20260806 03:50:50.610332 4751 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785988004637995-12279-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785988004637995-12279-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785988004637995-12279-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:50:50.626636 4751 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:50:50.626937 4751 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:50:50.627048 4751 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:50:50.631148 4751 rpc_server.cc:307] RPC server started. Bound to: 127.11.253.254:35281
I20260806 03:50:50.631194 4832 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.253.254:35281 every 8 connection(s)
I20260806 03:50:50.631567 4751 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785988004637995-12279-0/minicluster-data/master-0/data/info.pb
I20260806 03:50:50.632618 4833 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:50:50.632881 12279 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskToVWRe/build/release/bin/kudu as pid 4751
I20260806 03:50:50.633000 12279 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785988004637995-12279-0/minicluster-data/master-0/wal/instance
I20260806 03:50:50.633195 12279 external_mini_cluster.cc:1428] Setting key a7440cb4ccd6885c48853ed7cc54d708
I20260806 03:50:50.636433 4833 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 237c89cd5c5d4a50877ab4a0c0e27903: Bootstrap starting.
I20260806 03:50:50.637104 4833 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 237c89cd5c5d4a50877ab4a0c0e27903: Neither blocks nor log segments found. Creating new log.
I20260806 03:50:50.637422 4833 log.cc:824] T 00000000000000000000000000000000 P 237c89cd5c5d4a50877ab4a0c0e27903: Log is configured to *not* fsync() on all Append() calls
I20260806 03:50:50.638197 4833 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 237c89cd5c5d4a50877ab4a0c0e27903: No bootstrap required, opened a new log
I20260806 03:50:50.639425 4837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:50:50.633795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50442 (local address 127.11.253.254:35281)
0806 03:50:50.634035 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0806 03:50:50.634045 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 03:50:50.634083 (+ 38us) server_negotiation.cc:407] Connection header received
0806 03:50:50.634589 (+ 506us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:50:50.634599 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:50:50.634686 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:50:50.634852 (+ 166us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:50:50.635025 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:50:50.636392 (+ 1367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:50:50.637309 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:50:50.637807 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:50:50.637990 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:50:50.638787 (+ 797us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:50:50.638791 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:50:50.638791 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 03:50:50.638954 (+ 163us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:50:50.638992 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 03:50:50.639003 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:50:50.639076 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0806 03:50:50.639156 (+ 80us) server_negotiation.cc:299] Negotiation successful
0806 03:50:50.639235 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":95,"threads_started":1}
I20260806 03:50:50.640447 4833 raft_consensus.cc:348] T 00000000000000000000000000000000 P 237c89cd5c5d4a50877ab4a0c0e27903 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "237c89cd5c5d4a50877ab4a0c0e27903" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 35281 } }
I20260806 03:50:50.640606 4833 raft_consensus.cc:374] T 00000000000000000000000000000000 P 237c89cd5c5d4a50877ab4a0c0e27903 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:50:50.640654 4833 raft_consensus.cc:729] T 00000000000000000000000000000000 P 237c89cd5c5d4a50877ab4a0c0e27903 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 237c89cd5c5d4a50877ab4a0c0e27903, State: Initialized, Role: FOLLOWER
I20260806 03:50:50.640777 4833 consensus_queue.cc:260] T 00000000000000000000000000000000 P 237c89cd5c5d4a50877ab4a0c0e27903 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "237c89cd5c5d4a50877ab4a0c0e27903" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 35281 } }
I20260806 03:50:50.640882 4833 raft_consensus.cc:388] T 00000000000000000000000000000000 P 237c89cd5c5d4a50877ab4a0c0e27903 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 03:50:50.640926 4833 raft_consensus.cc:482] T 00000000000000000000000000000000 P 237c89cd5c5d4a50877ab4a0c0e27903 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 03:50:50.640976 4833 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 237c89cd5c5d4a50877ab4a0c0e27903 [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:50:50.641666 4833 raft_consensus.cc:504] T 00000000000000000000000000000000 P 237c89cd5c5d4a50877ab4a0c0e27903 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "237c89cd5c5d4a50877ab4a0c0e27903" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 35281 } }
I20260806 03:50:50.641809 4833 leader_election.cc:302] T 00000000000000000000000000000000 P 237c89cd5c5d4a50877ab4a0c0e27903 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 237c89cd5c5d4a50877ab4a0c0e27903; no voters:
I20260806 03:50:50.642071 4839 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 237c89cd5c5d4a50877ab4a0c0e27903 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 03:50:50.642017 4833 leader_election.cc:288] T 00000000000000000000000000000000 P 237c89cd5c5d4a50877ab4a0c0e27903 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 03:50:50.642230 4839 raft_consensus.cc:686] T 00000000000000000000000000000000 P 237c89cd5c5d4a50877ab4a0c0e27903 [term 1 LEADER]: Becoming Leader. State: Replica: 237c89cd5c5d4a50877ab4a0c0e27903, State: Running, Role: LEADER
I20260806 03:50:50.642367 4839 consensus_queue.cc:237] T 00000000000000000000000000000000 P 237c89cd5c5d4a50877ab4a0c0e27903 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "237c89cd5c5d4a50877ab4a0c0e27903" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 35281 } }
I20260806 03:50:50.642534 4833 sys_catalog.cc:564] T 00000000000000000000000000000000 P 237c89cd5c5d4a50877ab4a0c0e27903 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 03:50:50.642731 4841 sys_catalog.cc:455] T 00000000000000000000000000000000 P 237c89cd5c5d4a50877ab4a0c0e27903 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "237c89cd5c5d4a50877ab4a0c0e27903" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "237c89cd5c5d4a50877ab4a0c0e27903" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 35281 } } }
I20260806 03:50:50.642735 4840 sys_catalog.cc:455] T 00000000000000000000000000000000 P 237c89cd5c5d4a50877ab4a0c0e27903 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 237c89cd5c5d4a50877ab4a0c0e27903. Latest consensus state: current_term: 1 leader_uuid: "237c89cd5c5d4a50877ab4a0c0e27903" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "237c89cd5c5d4a50877ab4a0c0e27903" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 35281 } } }
I20260806 03:50:50.642853 4841 sys_catalog.cc:458] T 00000000000000000000000000000000 P 237c89cd5c5d4a50877ab4a0c0e27903 [sys.catalog]: This master's current role is: LEADER
I20260806 03:50:50.642861 4840 sys_catalog.cc:458] T 00000000000000000000000000000000 P 237c89cd5c5d4a50877ab4a0c0e27903 [sys.catalog]: This master's current role is: LEADER
I20260806 03:50:50.643087 4844 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 03:50:50.643635 4844 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 03:50:50.645200 4844 catalog_manager.cc:1347] Generated new cluster ID: 0b9aacca6fe64503a40ba64640f04aef
I20260806 03:50:50.645257 4844 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 03:50:50.656638 4844 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 03:50:50.657375 4844 catalog_manager.cc:1504] Loading token signing keys...
I20260806 03:50:50.669024 4844 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 237c89cd5c5d4a50877ab4a0c0e27903: Generated new TSK 0
I20260806 03:50:50.669251 4844 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 03:50:50.673365 12279 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskToVWRe/build/release/bin/kudu
/tmp/dist-test-taskToVWRe/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785988004637995-12279-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785988004637995-12279-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785988004637995-12279-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785988004637995-12279-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.253.193:0
--local_ip_for_outbound_sockets=127.11.253.193
--webserver_interface=127.11.253.193
--webserver_port=0
--tserver_master_addrs=127.11.253.254:35281
--builtin_ntp_servers=127.11.253.212:37747
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 03:50:50.758358 4865 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:50:50.758597 4865 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:50:50.758632 4865 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:50:50.759768 4865 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:50:50.759836 4865 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:50:50.759873 4865 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.253.193
I20260806 03:50:50.761240 4865 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.253.212:37747
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785988004637995-12279-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785988004637995-12279-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.253.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785988004637995-12279-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.253.193
--webserver_port=0
--tserver_master_addrs=127.11.253.254:35281
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4865
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.253.193
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785988004637995-12279-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:50:50.761508 4865 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:50:50.761823 4865 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:50:50.767575 4871 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:50:50.767624 4873 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:50:50.767743 4865 server_base.cc:1034] running on GCE node
W20260806 03:50:50.767598 4876 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:50:50.768085 4865 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:50:50.768392 4865 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:50:50.769574 4865 hybrid_clock.cc:648] HybridClock initialized: now 1785988250769549 us; error 42 us; skew 500 ppm
I20260806 03:50:50.771044 4865 webserver.cc:460] Webserver started at http://127.11.253.193:34583/ using document root <none> and password file <none>
I20260806 03:50:50.771378 4865 fs_manager.cc:359] Metadata directory not provided
I20260806 03:50:50.771456 4865 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:50:50.771621 4865 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:50:50.772718 4865 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785988004637995-12279-0/minicluster-data/ts-0/data/instance:
uuid: "5c5c18540913498ebc466583ae18de30"
format_stamp: "Formatted at 2026-08-06 03:50:50 on dist-test-slave-6ntk"
server_key: "f5b4f07902ed6be16fa114f4be527091"
server_key_iv: "3478b2dfb79ba0e5e158c3d2e6ea5ef6"
server_key_version: "encryptionkey@0"
I20260806 03:50:50.773090 4865 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785988004637995-12279-0/minicluster-data/ts-0/wal/instance:
uuid: "5c5c18540913498ebc466583ae18de30"
format_stamp: "Formatted at 2026-08-06 03:50:50 on dist-test-slave-6ntk"
server_key: "f5b4f07902ed6be16fa114f4be527091"
server_key_iv: "3478b2dfb79ba0e5e158c3d2e6ea5ef6"
server_key_version: "encryptionkey@0"
I20260806 03:50:50.774691 4865 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260806 03:50:50.775583 4886 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:50:50.775776 4865 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 03:50:50.775883 4865 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785988004637995-12279-0/minicluster-data/ts-0/data,/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785988004637995-12279-0/minicluster-data/ts-0/wal
uuid: "5c5c18540913498ebc466583ae18de30"
format_stamp: "Formatted at 2026-08-06 03:50:50 on dist-test-slave-6ntk"
server_key: "f5b4f07902ed6be16fa114f4be527091"
server_key_iv: "3478b2dfb79ba0e5e158c3d2e6ea5ef6"
server_key_version: "encryptionkey@0"
I20260806 03:50:50.775977 4865 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785988004637995-12279-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785988004637995-12279-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785988004637995-12279-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:50:50.785238 4865 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:50:50.785557 4865 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:50:50.785722 4865 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:50:50.786032 4865 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 03:50:50.786422 4865 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 03:50:50.786476 4865 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 03:50:50.786517 4865 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 03:50:50.786546 4865 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 03:50:50.792529 4865 rpc_server.cc:307] RPC server started. Bound to: 127.11.253.193:39265
I20260806 03:50:50.792620 5045 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.253.193:39265 every 8 connection(s)
I20260806 03:50:50.792954 4865 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785988004637995-12279-0/minicluster-data/ts-0/data/info.pb
I20260806 03:50:50.798766 4837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:50:50.793852 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.253.193:44649 (local address 127.11.253.254:35281)
0806 03:50:50.793957 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0806 03:50:50.793959 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:50:50.794229 (+ 270us) server_negotiation.cc:407] Connection header received
0806 03:50:50.794746 (+ 517us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:50:50.794747 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:50:50.794761 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:50:50.794828 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:50:50.795607 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:50:50.796134 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:50:50.797227 (+ 1093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:50:50.797440 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:50:50.797505 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:50:50.798442 (+ 937us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:50:50.798444 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:50:50.798445 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 03:50:50.798450 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:50:50.798467 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 03:50:50.798472 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:50:50.798549 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0806 03:50:50.798638 (+ 89us) server_negotiation.cc:299] Negotiation successful
0806 03:50:50.798676 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260806 03:50:50.799029 5048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:50:50.793830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:35281 (local address 127.11.253.193:44649)
0806 03:50:50.794080 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0806 03:50:50.794125 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 03:50:50.794648 (+ 523us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:50:50.794913 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:50:50.794917 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:50:50.794926 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:50:50.795478 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:50:50.795482 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:50:50.796221 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:50:50.796222 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:50:50.797114 (+ 892us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:50:50.797116 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:50:50.797528 (+ 412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:50:50.797529 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:50:50.797650 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:50:50.798263 (+ 613us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 03:50:50.798263 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:50:50.798302 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 03:50:50.798332 (+ 30us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:50:50.798549 (+ 217us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:50:50.798557 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:50:50.798560 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 03:50:50.798631 (+ 71us) client_negotiation.cc:240] Negotiation successful
0806 03:50:50.798746 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":142,"threads_started":1}
I20260806 03:50:50.799574 5046 heartbeater.cc:347] Connected to a master server at 127.11.253.254:35281
I20260806 03:50:50.799683 5046 heartbeater.cc:464] Registering TS with master...
I20260806 03:50:50.799721 12279 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskToVWRe/build/release/bin/kudu as pid 4865
I20260806 03:50:50.799856 12279 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785988004637995-12279-0/minicluster-data/ts-0/wal/instance
I20260806 03:50:50.799949 5046 heartbeater.cc:511] Master 127.11.253.254:35281 requested a full tablet report, sending...
I20260806 03:50:50.800114 12279 external_mini_cluster.cc:1428] Setting key df9eda5328c741cb458b3ede94785abb
I20260806 03:50:50.800587 4791 ts_manager.cc:194] Registered new tserver with Master: 5c5c18540913498ebc466583ae18de30 (127.11.253.193:39265)
I20260806 03:50:50.801345 4791 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.11.253.193:44649
I20260806 03:50:50.801465 12279 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskToVWRe/build/release/bin/kudu
/tmp/dist-test-taskToVWRe/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785988004637995-12279-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785988004637995-12279-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785988004637995-12279-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785988004637995-12279-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.253.194:0
--local_ip_for_outbound_sockets=127.11.253.194
--webserver_interface=127.11.253.194
--webserver_port=0
--tserver_master_addrs=127.11.253.254:35281
--builtin_ntp_servers=127.11.253.212:37747
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 03:50:50.886169 5049 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:50:50.886390 5049 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:50:50.886412 5049 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:50:50.887494 5049 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:50:50.887530 5049 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:50:50.887616 5049 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.253.194
I20260806 03:50:50.889008 5049 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.253.212:37747
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785988004637995-12279-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785988004637995-12279-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.253.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785988004637995-12279-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.253.194
--webserver_port=0
--tserver_master_addrs=127.11.253.254:35281
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5049
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.253.194
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785988004637995-12279-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:50:50.889233 5049 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:50:50.889504 5049 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:50:50.894819 5055 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:50:50.894863 5058 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:50:50.894873 5056 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:50:50.895741 5049 server_base.cc:1034] running on GCE node
I20260806 03:50:50.895948 5049 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:50:50.896193 5049 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:50:50.897344 5049 hybrid_clock.cc:648] HybridClock initialized: now 1785988250897326 us; error 36 us; skew 500 ppm
I20260806 03:50:50.898869 5049 webserver.cc:460] Webserver started at http://127.11.253.194:39359/ using document root <none> and password file <none>
I20260806 03:50:50.899137 5049 fs_manager.cc:359] Metadata directory not provided
I20260806 03:50:50.899209 5049 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:50:50.899408 5049 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:50:50.900468 5049 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785988004637995-12279-0/minicluster-data/ts-1/data/instance:
uuid: "daef68b474584e4ba68d5396c0af4a9f"
format_stamp: "Formatted at 2026-08-06 03:50:50 on dist-test-slave-6ntk"
server_key: "88fd2bf14a3348472631d4816ced690a"
server_key_iv: "9865bc6dc59d2848fff58d28fb97d86b"
server_key_version: "encryptionkey@0"
I20260806 03:50:50.900843 5049 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785988004637995-12279-0/minicluster-data/ts-1/wal/instance:
uuid: "daef68b474584e4ba68d5396c0af4a9f"
format_stamp: "Formatted at 2026-08-06 03:50:50 on dist-test-slave-6ntk"
server_key: "88fd2bf14a3348472631d4816ced690a"
server_key_iv: "9865bc6dc59d2848fff58d28fb97d86b"
server_key_version: "encryptionkey@0"
I20260806 03:50:50.902253 5049 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 03:50:50.903059 5066 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:50:50.903265 5049 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260806 03:50:50.903375 5049 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785988004637995-12279-0/minicluster-data/ts-1/data,/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785988004637995-12279-0/minicluster-data/ts-1/wal
uuid: "daef68b474584e4ba68d5396c0af4a9f"
format_stamp: "Formatted at 2026-08-06 03:50:50 on dist-test-slave-6ntk"
server_key: "88fd2bf14a3348472631d4816ced690a"
server_key_iv: "9865bc6dc59d2848fff58d28fb97d86b"
server_key_version: "encryptionkey@0"
I20260806 03:50:50.903451 5049 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785988004637995-12279-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785988004637995-12279-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785988004637995-12279-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:50:50.915755 5049 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:50:50.916071 5049 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:50:50.916234 5049 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:50:50.916463 5049 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 03:50:50.916806 5049 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 03:50:50.916859 5049 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 03:50:50.916894 5049 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 03:50:50.916921 5049 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 03:50:50.922778 5049 rpc_server.cc:307] RPC server started. Bound to: 127.11.253.194:37949
I20260806 03:50:50.922845 5216 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.253.194:37949 every 8 connection(s)
I20260806 03:50:50.923168 5049 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785988004637995-12279-0/minicluster-data/ts-1/data/info.pb
I20260806 03:50:50.927479 12279 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskToVWRe/build/release/bin/kudu as pid 5049
I20260806 03:50:50.927589 12279 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785988004637995-12279-0/minicluster-data/ts-1/wal/instance
I20260806 03:50:50.927805 12279 external_mini_cluster.cc:1428] Setting key a2d701db6019626d0c1bfeab46c74320
I20260806 03:50:50.928934 12279 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskToVWRe/build/release/bin/kudu
/tmp/dist-test-taskToVWRe/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785988004637995-12279-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785988004637995-12279-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785988004637995-12279-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785988004637995-12279-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.253.195:0
--local_ip_for_outbound_sockets=127.11.253.195
--webserver_interface=127.11.253.195
--webserver_port=0
--tserver_master_addrs=127.11.253.254:35281
--builtin_ntp_servers=127.11.253.212:37747
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260806 03:50:50.928956 4837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:50:50.923960 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.253.194:36299 (local address 127.11.253.254:35281)
0806 03:50:50.924024 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0806 03:50:50.924026 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:50:50.924275 (+ 249us) server_negotiation.cc:407] Connection header received
0806 03:50:50.924868 (+ 593us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:50:50.924869 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:50:50.924883 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:50:50.924930 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:50:50.925616 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:50:50.926202 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:50:50.927180 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:50:50.927480 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:50:50.927557 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:50:50.928681 (+ 1124us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:50:50.928683 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:50:50.928683 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 03:50:50.928688 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:50:50.928704 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 03:50:50.928709 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:50:50.928785 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0806 03:50:50.928836 (+ 51us) server_negotiation.cc:299] Negotiation successful
0806 03:50:50.928862 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260806 03:50:50.929123 5222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:50:50.923947 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:35281 (local address 127.11.253.194:36299)
0806 03:50:50.924156 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0806 03:50:50.924189 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0806 03:50:50.924740 (+ 551us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:50:50.925012 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:50:50.925016 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:50:50.925024 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:50:50.925520 (+ 496us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:50:50.925524 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:50:50.926313 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:50:50.926314 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:50:50.927078 (+ 764us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:50:50.927080 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:50:50.927609 (+ 529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:50:50.927611 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:50:50.927800 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:50:50.928518 (+ 718us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 03:50:50.928518 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:50:50.928559 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 03:50:50.928589 (+ 30us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:50:50.928777 (+ 188us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:50:50.928779 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:50:50.928782 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 03:50:50.928825 (+ 43us) client_negotiation.cc:240] Negotiation successful
0806 03:50:50.928904 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":97,"threads_started":1}
I20260806 03:50:50.929601 5217 heartbeater.cc:347] Connected to a master server at 127.11.253.254:35281
I20260806 03:50:50.929697 5217 heartbeater.cc:464] Registering TS with master...
I20260806 03:50:50.929961 5217 heartbeater.cc:511] Master 127.11.253.254:35281 requested a full tablet report, sending...
I20260806 03:50:50.930364 4791 ts_manager.cc:194] Registered new tserver with Master: daef68b474584e4ba68d5396c0af4a9f (127.11.253.194:37949)
I20260806 03:50:50.930759 4791 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.11.253.194:36299
W20260806 03:50:51.015385 5223 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:50:51.015614 5223 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:50:51.015635 5223 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:50:51.016829 5223 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:50:51.016875 5223 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:50:51.016902 5223 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.253.195
I20260806 03:50:51.018384 5223 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.253.212:37747
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785988004637995-12279-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785988004637995-12279-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.253.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785988004637995-12279-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.253.195
--webserver_port=0
--tserver_master_addrs=127.11.253.254:35281
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5223
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.253.195
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785988004637995-12279-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:50:51.018656 5223 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:50:51.018977 5223 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:50:51.024443 5228 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:50:51.024457 5230 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:50:51.024464 5232 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:50:51.024803 5223 server_base.cc:1034] running on GCE node
I20260806 03:50:51.025038 5223 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:50:51.025319 5223 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:50:51.026515 5223 hybrid_clock.cc:648] HybridClock initialized: now 1785988251026496 us; error 48 us; skew 500 ppm
I20260806 03:50:51.028195 5223 webserver.cc:460] Webserver started at http://127.11.253.195:41647/ using document root <none> and password file <none>
I20260806 03:50:51.028419 5223 fs_manager.cc:359] Metadata directory not provided
I20260806 03:50:51.028465 5223 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:50:51.028568 5223 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:50:51.029698 5223 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785988004637995-12279-0/minicluster-data/ts-2/data/instance:
uuid: "882521371a464d22b968b8863e2e7ba1"
format_stamp: "Formatted at 2026-08-06 03:50:51 on dist-test-slave-6ntk"
server_key: "45c8df11ba6c71c66321fa24869721be"
server_key_iv: "bd3895f3fa87d18666befbafdc06cd24"
server_key_version: "encryptionkey@0"
I20260806 03:50:51.030130 5223 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785988004637995-12279-0/minicluster-data/ts-2/wal/instance:
uuid: "882521371a464d22b968b8863e2e7ba1"
format_stamp: "Formatted at 2026-08-06 03:50:51 on dist-test-slave-6ntk"
server_key: "45c8df11ba6c71c66321fa24869721be"
server_key_iv: "bd3895f3fa87d18666befbafdc06cd24"
server_key_version: "encryptionkey@0"
I20260806 03:50:51.031528 5223 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 03:50:51.032285 5238 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:50:51.032464 5223 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 03:50:51.032553 5223 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785988004637995-12279-0/minicluster-data/ts-2/data,/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785988004637995-12279-0/minicluster-data/ts-2/wal
uuid: "882521371a464d22b968b8863e2e7ba1"
format_stamp: "Formatted at 2026-08-06 03:50:51 on dist-test-slave-6ntk"
server_key: "45c8df11ba6c71c66321fa24869721be"
server_key_iv: "bd3895f3fa87d18666befbafdc06cd24"
server_key_version: "encryptionkey@0"
I20260806 03:50:51.032608 5223 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785988004637995-12279-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785988004637995-12279-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785988004637995-12279-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:50:51.044945 5223 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:50:51.045222 5223 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:50:51.045387 5223 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:50:51.045636 5223 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 03:50:51.046051 5223 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 03:50:51.046089 5223 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 03:50:51.046113 5223 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 03:50:51.046159 5223 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 03:50:51.051848 5223 rpc_server.cc:307] RPC server started. Bound to: 127.11.253.195:44629
I20260806 03:50:51.051913 5385 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.253.195:44629 every 8 connection(s)
I20260806 03:50:51.052309 5223 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785988004637995-12279-0/minicluster-data/ts-2/data/info.pb
I20260806 03:50:51.055601 12279 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskToVWRe/build/release/bin/kudu as pid 5223
I20260806 03:50:51.055698 12279 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785988004637995-12279-0/minicluster-data/ts-2/wal/instance
I20260806 03:50:51.055898 12279 external_mini_cluster.cc:1428] Setting key 6fe2f53b90465bec490bd00eacbd0b94
I20260806 03:50:51.058490 4837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:50:51.053188 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.253.195:42013 (local address 127.11.253.254:35281)
0806 03:50:51.053302 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0806 03:50:51.053304 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:50:51.053516 (+ 212us) server_negotiation.cc:407] Connection header received
0806 03:50:51.054090 (+ 574us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:50:51.054091 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:50:51.054104 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:50:51.054167 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:50:51.054931 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:50:51.055498 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:50:51.056855 (+ 1357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:50:51.057060 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:50:51.057152 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:50:51.058106 (+ 954us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:50:51.058108 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:50:51.058108 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 03:50:51.058113 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:50:51.058138 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 03:50:51.058142 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:50:51.058210 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0806 03:50:51.058368 (+ 158us) server_negotiation.cc:299] Negotiation successful
0806 03:50:51.058402 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260806 03:50:51.058766 5388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:50:51.053136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:35281 (local address 127.11.253.195:42013)
0806 03:50:51.053384 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0806 03:50:51.053420 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0806 03:50:51.053980 (+ 560us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:50:51.054288 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:50:51.054292 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:50:51.054301 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:50:51.054830 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:50:51.054834 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:50:51.055616 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:50:51.055617 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:50:51.056768 (+ 1151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:50:51.056771 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:50:51.057164 (+ 393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:50:51.057165 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:50:51.057313 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:50:51.057917 (+ 604us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 03:50:51.057918 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:50:51.057951 (+ 33us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 03:50:51.057977 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:50:51.058212 (+ 235us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:50:51.058219 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:50:51.058223 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 03:50:51.058368 (+ 145us) client_negotiation.cc:240] Negotiation successful
0806 03:50:51.058484 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":99,"threads_started":1}
I20260806 03:50:51.059319 5386 heartbeater.cc:347] Connected to a master server at 127.11.253.254:35281
I20260806 03:50:51.059420 5386 heartbeater.cc:464] Registering TS with master...
I20260806 03:50:51.059571 5386 heartbeater.cc:511] Master 127.11.253.254:35281 requested a full tablet report, sending...
I20260806 03:50:51.059937 4791 ts_manager.cc:194] Registered new tserver with Master: 882521371a464d22b968b8863e2e7ba1 (127.11.253.195:44629)
I20260806 03:50:51.060328 4791 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.11.253.195:42013
I20260806 03:50:51.068097 12279 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 03:50:51.072963 4837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:50:51.069695 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50458 (local address 127.11.253.254:35281)
0806 03:50:51.069783 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0806 03:50:51.069785 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:50:51.069914 (+ 129us) server_negotiation.cc:407] Connection header received
0806 03:50:51.070021 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:50:51.070022 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:50:51.070034 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:50:51.070083 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:50:51.070291 (+ 208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:50:51.070775 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:50:51.071580 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:50:51.071786 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:50:51.071867 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:50:51.072636 (+ 769us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:50:51.072638 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:50:51.072638 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 03:50:51.072643 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:50:51.072664 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 03:50:51.072669 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:50:51.072735 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0806 03:50:51.072831 (+ 96us) server_negotiation.cc:299] Negotiation successful
0806 03:50:51.072856 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260806 03:50:51.074838 4791 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:50458:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 03:50:51.076574 4791 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 03:50:51.083792 5396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:50:51.079166 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.194:37949 (local address 127.0.0.1:33018)
0806 03:50:51.080088 (+ 922us) negotiation.cc:107] Waiting for socket to connect
0806 03:50:51.080100 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 03:50:51.080168 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:50:51.080344 (+ 176us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:50:51.080344 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:50:51.080347 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 03:50:51.080379 (+ 32us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:50:51.080380 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:50:51.081609 (+ 1229us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:50:51.081610 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:50:51.082564 (+ 954us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:50:51.082566 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:50:51.083434 (+ 868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:50:51.083435 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:50:51.083527 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:50:51.083547 (+ 20us) client_negotiation.cc:769] Sending connection context
0806 03:50:51.083621 (+ 74us) client_negotiation.cc:240] Negotiation successful
0806 03:50:51.083682 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":907,"spinlock_wait_cycles":1280,"thread_start_us":211,"threads_started":1}
I20260806 03:50:51.083886 5401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:50:51.079467 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36744 (local address 127.11.253.193:39265)
0806 03:50:51.079676 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0806 03:50:51.079682 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 03:50:51.079697 (+ 15us) server_negotiation.cc:407] Connection header received
0806 03:50:51.079746 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:50:51.079749 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:50:51.079767 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:50:51.079873 (+ 106us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 03:50:51.080275 (+ 402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:50:51.081140 (+ 865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:50:51.082828 (+ 1688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:50:51.083507 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:50:51.083602 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:50:51.083698 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0806 03:50:51.083741 (+ 43us) server_negotiation.cc:299] Negotiation successful
0806 03:50:51.083781 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":167,"thread_start_us":131,"threads_started":1}
I20260806 03:50:51.084079 5399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:50:51.079165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.195:44629 (local address 127.0.0.1:33042)
0806 03:50:51.079696 (+ 531us) negotiation.cc:107] Waiting for socket to connect
0806 03:50:51.079704 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0806 03:50:51.079745 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:50:51.080085 (+ 340us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:50:51.080091 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:50:51.080094 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 03:50:51.080214 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:50:51.080216 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:50:51.081590 (+ 1374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:50:51.081591 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:50:51.082858 (+ 1267us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:50:51.082860 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:50:51.083854 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:50:51.083854 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:50:51.083914 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:50:51.083919 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 03:50:51.083975 (+ 56us) client_negotiation.cc:240] Negotiation successful
0806 03:50:51.084019 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":508,"mutex_wait_us":65,"thread_start_us":208,"threads_started":1}
I20260806 03:50:51.084194 5403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:50:51.079821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33018 (local address 127.11.253.194:37949)
0806 03:50:51.080030 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0806 03:50:51.080034 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 03:50:51.080190 (+ 156us) server_negotiation.cc:407] Connection header received
0806 03:50:51.080242 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:50:51.080244 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:50:51.080269 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:50:51.080350 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 03:50:51.080433 (+ 83us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:50:51.081516 (+ 1083us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:50:51.082661 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:50:51.083335 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:50:51.083415 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:50:51.083517 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0806 03:50:51.084029 (+ 512us) server_negotiation.cc:299] Negotiation successful
0806 03:50:51.084087 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":178,"thread_start_us":98,"threads_started":1}
I20260806 03:50:51.084332 5402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:50:51.079546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33042 (local address 127.11.253.195:44629)
0806 03:50:51.079834 (+ 288us) server_negotiation.cc:206] Beginning negotiation
0806 03:50:51.079839 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 03:50:51.079849 (+ 10us) server_negotiation.cc:407] Connection header received
0806 03:50:51.079894 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:50:51.079896 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:50:51.079912 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:50:51.079995 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 03:50:51.080311 (+ 316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:50:51.081489 (+ 1178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:50:51.082949 (+ 1460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:50:51.083670 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:50:51.083741 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:50:51.083821 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0806 03:50:51.084183 (+ 362us) server_negotiation.cc:299] Negotiation successful
0806 03:50:51.084223 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":185,"thread_start_us":107,"threads_started":1}
I20260806 03:50:51.084966 5398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:50:51.079102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.193:39265 (local address 127.0.0.1:36744)
0806 03:50:51.079450 (+ 348us) negotiation.cc:107] Waiting for socket to connect
0806 03:50:51.079497 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0806 03:50:51.079572 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:50:51.080083 (+ 511us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:50:51.080087 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:50:51.080089 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 03:50:51.080194 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:50:51.080197 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:50:51.081242 (+ 1045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:50:51.081244 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:50:51.082720 (+ 1476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:50:51.082723 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:50:51.083601 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:50:51.083602 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:50:51.083667 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:50:51.083671 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 03:50:51.083715 (+ 44us) client_negotiation.cc:240] Negotiation successful
0806 03:50:51.083767 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":313,"mutex_wait_us":38,"thread_start_us":86,"threads_started":1}
I20260806 03:50:51.085000 5302 tablet_service.cc:1467] Processing CreateTablet for tablet ef8fc29bc5c840a0ad555f1902ec6348 (DEFAULT_TABLE table=test-table [id=6be08c1105db409a97e41ab39e14c5ab]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 03:50:51.084975 5134 tablet_service.cc:1467] Processing CreateTablet for tablet ef8fc29bc5c840a0ad555f1902ec6348 (DEFAULT_TABLE table=test-table [id=6be08c1105db409a97e41ab39e14c5ab]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 03:50:51.085419 5134 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ef8fc29bc5c840a0ad555f1902ec6348. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:50:51.085438 5302 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ef8fc29bc5c840a0ad555f1902ec6348. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:50:51.085903 4946 tablet_service.cc:1467] Processing CreateTablet for tablet ef8fc29bc5c840a0ad555f1902ec6348 (DEFAULT_TABLE table=test-table [id=6be08c1105db409a97e41ab39e14c5ab]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 03:50:51.086294 4946 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ef8fc29bc5c840a0ad555f1902ec6348. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:50:51.088701 5404 tablet_bootstrap.cc:492] T ef8fc29bc5c840a0ad555f1902ec6348 P daef68b474584e4ba68d5396c0af4a9f: Bootstrap starting.
I20260806 03:50:51.089331 5405 tablet_bootstrap.cc:492] T ef8fc29bc5c840a0ad555f1902ec6348 P 882521371a464d22b968b8863e2e7ba1: Bootstrap starting.
I20260806 03:50:51.089516 5404 tablet_bootstrap.cc:654] T ef8fc29bc5c840a0ad555f1902ec6348 P daef68b474584e4ba68d5396c0af4a9f: Neither blocks nor log segments found. Creating new log.
I20260806 03:50:51.089901 5404 log.cc:824] T ef8fc29bc5c840a0ad555f1902ec6348 P daef68b474584e4ba68d5396c0af4a9f: Log is configured to *not* fsync() on all Append() calls
I20260806 03:50:51.090039 5405 tablet_bootstrap.cc:654] T ef8fc29bc5c840a0ad555f1902ec6348 P 882521371a464d22b968b8863e2e7ba1: Neither blocks nor log segments found. Creating new log.
I20260806 03:50:51.090204 5406 tablet_bootstrap.cc:492] T ef8fc29bc5c840a0ad555f1902ec6348 P 5c5c18540913498ebc466583ae18de30: Bootstrap starting.
I20260806 03:50:51.090384 5405 log.cc:824] T ef8fc29bc5c840a0ad555f1902ec6348 P 882521371a464d22b968b8863e2e7ba1: Log is configured to *not* fsync() on all Append() calls
I20260806 03:50:51.090740 5404 tablet_bootstrap.cc:492] T ef8fc29bc5c840a0ad555f1902ec6348 P daef68b474584e4ba68d5396c0af4a9f: No bootstrap required, opened a new log
I20260806 03:50:51.090831 5404 ts_tablet_manager.cc:1397] T ef8fc29bc5c840a0ad555f1902ec6348 P daef68b474584e4ba68d5396c0af4a9f: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 03:50:51.090920 5406 tablet_bootstrap.cc:654] T ef8fc29bc5c840a0ad555f1902ec6348 P 5c5c18540913498ebc466583ae18de30: Neither blocks nor log segments found. Creating new log.
I20260806 03:50:51.091172 5405 tablet_bootstrap.cc:492] T ef8fc29bc5c840a0ad555f1902ec6348 P 882521371a464d22b968b8863e2e7ba1: No bootstrap required, opened a new log
I20260806 03:50:51.091240 5405 ts_tablet_manager.cc:1397] T ef8fc29bc5c840a0ad555f1902ec6348 P 882521371a464d22b968b8863e2e7ba1: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260806 03:50:51.091498 5406 log.cc:824] T ef8fc29bc5c840a0ad555f1902ec6348 P 5c5c18540913498ebc466583ae18de30: Log is configured to *not* fsync() on all Append() calls
I20260806 03:50:51.092445 5406 tablet_bootstrap.cc:492] T ef8fc29bc5c840a0ad555f1902ec6348 P 5c5c18540913498ebc466583ae18de30: No bootstrap required, opened a new log
I20260806 03:50:51.092522 5406 ts_tablet_manager.cc:1397] T ef8fc29bc5c840a0ad555f1902ec6348 P 5c5c18540913498ebc466583ae18de30: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 03:50:51.093449 5405 raft_consensus.cc:348] T ef8fc29bc5c840a0ad555f1902ec6348 P 882521371a464d22b968b8863e2e7ba1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c5c18540913498ebc466583ae18de30" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 39265 } } peers { permanent_uuid: "882521371a464d22b968b8863e2e7ba1" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 44629 } } peers { permanent_uuid: "daef68b474584e4ba68d5396c0af4a9f" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 37949 } }
I20260806 03:50:51.093611 5405 raft_consensus.cc:374] T ef8fc29bc5c840a0ad555f1902ec6348 P 882521371a464d22b968b8863e2e7ba1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:50:51.093654 5405 raft_consensus.cc:729] T ef8fc29bc5c840a0ad555f1902ec6348 P 882521371a464d22b968b8863e2e7ba1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 882521371a464d22b968b8863e2e7ba1, State: Initialized, Role: FOLLOWER
I20260806 03:50:51.093771 5405 consensus_queue.cc:260] T ef8fc29bc5c840a0ad555f1902ec6348 P 882521371a464d22b968b8863e2e7ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c5c18540913498ebc466583ae18de30" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 39265 } } peers { permanent_uuid: "882521371a464d22b968b8863e2e7ba1" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 44629 } } peers { permanent_uuid: "daef68b474584e4ba68d5396c0af4a9f" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 37949 } }
I20260806 03:50:51.093900 5404 raft_consensus.cc:348] T ef8fc29bc5c840a0ad555f1902ec6348 P daef68b474584e4ba68d5396c0af4a9f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c5c18540913498ebc466583ae18de30" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 39265 } } peers { permanent_uuid: "882521371a464d22b968b8863e2e7ba1" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 44629 } } peers { permanent_uuid: "daef68b474584e4ba68d5396c0af4a9f" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 37949 } }
I20260806 03:50:51.094079 5404 raft_consensus.cc:374] T ef8fc29bc5c840a0ad555f1902ec6348 P daef68b474584e4ba68d5396c0af4a9f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:50:51.094064 5405 ts_tablet_manager.cc:1428] T ef8fc29bc5c840a0ad555f1902ec6348 P 882521371a464d22b968b8863e2e7ba1: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260806 03:50:51.094125 5404 raft_consensus.cc:729] T ef8fc29bc5c840a0ad555f1902ec6348 P daef68b474584e4ba68d5396c0af4a9f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: daef68b474584e4ba68d5396c0af4a9f, State: Initialized, Role: FOLLOWER
I20260806 03:50:51.094213 5386 heartbeater.cc:503] Master 127.11.253.254:35281 was elected leader, sending a full tablet report...
I20260806 03:50:51.094267 5404 consensus_queue.cc:260] T ef8fc29bc5c840a0ad555f1902ec6348 P daef68b474584e4ba68d5396c0af4a9f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c5c18540913498ebc466583ae18de30" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 39265 } } peers { permanent_uuid: "882521371a464d22b968b8863e2e7ba1" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 44629 } } peers { permanent_uuid: "daef68b474584e4ba68d5396c0af4a9f" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 37949 } }
I20260806 03:50:51.094549 5404 ts_tablet_manager.cc:1428] T ef8fc29bc5c840a0ad555f1902ec6348 P daef68b474584e4ba68d5396c0af4a9f: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260806 03:50:51.094691 5217 heartbeater.cc:503] Master 127.11.253.254:35281 was elected leader, sending a full tablet report...
I20260806 03:50:51.095595 5406 raft_consensus.cc:348] T ef8fc29bc5c840a0ad555f1902ec6348 P 5c5c18540913498ebc466583ae18de30 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c5c18540913498ebc466583ae18de30" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 39265 } } peers { permanent_uuid: "882521371a464d22b968b8863e2e7ba1" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 44629 } } peers { permanent_uuid: "daef68b474584e4ba68d5396c0af4a9f" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 37949 } }
I20260806 03:50:51.095808 5406 raft_consensus.cc:374] T ef8fc29bc5c840a0ad555f1902ec6348 P 5c5c18540913498ebc466583ae18de30 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:50:51.095866 5406 raft_consensus.cc:729] T ef8fc29bc5c840a0ad555f1902ec6348 P 5c5c18540913498ebc466583ae18de30 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c5c18540913498ebc466583ae18de30, State: Initialized, Role: FOLLOWER
I20260806 03:50:51.096010 5406 consensus_queue.cc:260] T ef8fc29bc5c840a0ad555f1902ec6348 P 5c5c18540913498ebc466583ae18de30 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c5c18540913498ebc466583ae18de30" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 39265 } } peers { permanent_uuid: "882521371a464d22b968b8863e2e7ba1" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 44629 } } peers { permanent_uuid: "daef68b474584e4ba68d5396c0af4a9f" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 37949 } }
I20260806 03:50:51.096246 5406 ts_tablet_manager.cc:1428] T ef8fc29bc5c840a0ad555f1902ec6348 P 5c5c18540913498ebc466583ae18de30: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260806 03:50:51.096351 5046 heartbeater.cc:503] Master 127.11.253.254:35281 was elected leader, sending a full tablet report...
W20260806 03:50:51.173811 5221 tablet.cc:2369] T ef8fc29bc5c840a0ad555f1902ec6348 P daef68b474584e4ba68d5396c0af4a9f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 03:50:51.265101 5411 raft_consensus.cc:482] T ef8fc29bc5c840a0ad555f1902ec6348 P daef68b474584e4ba68d5396c0af4a9f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 03:50:51.265281 5411 raft_consensus.cc:504] T ef8fc29bc5c840a0ad555f1902ec6348 P daef68b474584e4ba68d5396c0af4a9f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c5c18540913498ebc466583ae18de30" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 39265 } } peers { permanent_uuid: "882521371a464d22b968b8863e2e7ba1" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 44629 } } peers { permanent_uuid: "daef68b474584e4ba68d5396c0af4a9f" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 37949 } }
I20260806 03:50:51.265708 5411 leader_election.cc:288] T ef8fc29bc5c840a0ad555f1902ec6348 P daef68b474584e4ba68d5396c0af4a9f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5c5c18540913498ebc466583ae18de30 (127.11.253.193:39265), 882521371a464d22b968b8863e2e7ba1 (127.11.253.195:44629)
I20260806 03:50:51.268584 5222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:50:51.265922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.195:44629 (local address 127.11.253.194:38327)
0806 03:50:51.266001 (+ 79us) negotiation.cc:107] Waiting for socket to connect
0806 03:50:51.266015 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 03:50:51.266060 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:50:51.266205 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:50:51.266206 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:50:51.266208 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 03:50:51.266288 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:50:51.266290 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:50:51.266946 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:50:51.266947 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:50:51.267692 (+ 745us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:50:51.267694 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:50:51.268316 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:50:51.268317 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:50:51.268393 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:50:51.268396 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 03:50:51.268463 (+ 67us) client_negotiation.cc:240] Negotiation successful
0806 03:50:51.268506 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":60}
I20260806 03:50:51.268586 5402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:50:51.266041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.253.194:38327 (local address 127.11.253.195:44629)
0806 03:50:51.266092 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0806 03:50:51.266093 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 03:50:51.266101 (+ 8us) server_negotiation.cc:407] Connection header received
0806 03:50:51.266133 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:50:51.266134 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:50:51.266142 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:50:51.266201 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 03:50:51.266373 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:50:51.266860 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:50:51.267783 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:50:51.268216 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:50:51.268316 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:50:51.268325 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0806 03:50:51.268469 (+ 144us) server_negotiation.cc:299] Negotiation successful
0806 03:50:51.268507 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260806 03:50:51.268770 5401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:50:51.265997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.253.194:33931 (local address 127.11.253.193:39265)
0806 03:50:51.266049 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0806 03:50:51.266052 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 03:50:51.266201 (+ 149us) server_negotiation.cc:407] Connection header received
0806 03:50:51.266287 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:50:51.266289 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:50:51.266301 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:50:51.266363 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 03:50:51.266534 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:50:51.267008 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:50:51.267940 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:50:51.268332 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:50:51.268419 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:50:51.268427 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0806 03:50:51.268596 (+ 169us) server_negotiation.cc:299] Negotiation successful
0806 03:50:51.268667 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260806 03:50:51.269024 5413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:50:51.265948 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.193:39265 (local address 127.11.253.194:33931)
0806 03:50:51.266125 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0806 03:50:51.266134 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 03:50:51.266214 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:50:51.266374 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:50:51.266375 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:50:51.266377 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 03:50:51.266458 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:50:51.266459 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:50:51.267098 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:50:51.267099 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:50:51.267848 (+ 749us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:50:51.267850 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:50:51.268419 (+ 569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:50:51.268420 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:50:51.268491 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:50:51.268493 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 03:50:51.268884 (+ 391us) client_negotiation.cc:240] Negotiation successful
0806 03:50:51.268925 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":106,"threads_started":1}
I20260806 03:50:51.268988 5328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef8fc29bc5c840a0ad555f1902ec6348" candidate_uuid: "daef68b474584e4ba68d5396c0af4a9f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "882521371a464d22b968b8863e2e7ba1" is_pre_election: true
I20260806 03:50:51.269147 5328 raft_consensus.cc:2454] T ef8fc29bc5c840a0ad555f1902ec6348 P 882521371a464d22b968b8863e2e7ba1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate daef68b474584e4ba68d5396c0af4a9f in term 0.
I20260806 03:50:51.269367 5070 leader_election.cc:302] T ef8fc29bc5c840a0ad555f1902ec6348 P daef68b474584e4ba68d5396c0af4a9f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 882521371a464d22b968b8863e2e7ba1, daef68b474584e4ba68d5396c0af4a9f; no voters:
I20260806 03:50:51.269408 4976 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef8fc29bc5c840a0ad555f1902ec6348" candidate_uuid: "daef68b474584e4ba68d5396c0af4a9f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c5c18540913498ebc466583ae18de30" is_pre_election: true
I20260806 03:50:51.269497 5411 raft_consensus.cc:2781] T ef8fc29bc5c840a0ad555f1902ec6348 P daef68b474584e4ba68d5396c0af4a9f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 03:50:51.269547 5411 raft_consensus.cc:482] T ef8fc29bc5c840a0ad555f1902ec6348 P daef68b474584e4ba68d5396c0af4a9f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 03:50:51.269577 5411 raft_consensus.cc:3037] T ef8fc29bc5c840a0ad555f1902ec6348 P daef68b474584e4ba68d5396c0af4a9f [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:50:51.269582 4976 raft_consensus.cc:2454] T ef8fc29bc5c840a0ad555f1902ec6348 P 5c5c18540913498ebc466583ae18de30 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate daef68b474584e4ba68d5396c0af4a9f in term 0.
I20260806 03:50:51.270426 5411 raft_consensus.cc:504] T ef8fc29bc5c840a0ad555f1902ec6348 P daef68b474584e4ba68d5396c0af4a9f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c5c18540913498ebc466583ae18de30" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 39265 } } peers { permanent_uuid: "882521371a464d22b968b8863e2e7ba1" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 44629 } } peers { permanent_uuid: "daef68b474584e4ba68d5396c0af4a9f" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 37949 } }
I20260806 03:50:51.270586 5411 leader_election.cc:288] T ef8fc29bc5c840a0ad555f1902ec6348 P daef68b474584e4ba68d5396c0af4a9f [CANDIDATE]: Term 1 election: Requested vote from peers 5c5c18540913498ebc466583ae18de30 (127.11.253.193:39265), 882521371a464d22b968b8863e2e7ba1 (127.11.253.195:44629)
I20260806 03:50:51.270682 4976 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef8fc29bc5c840a0ad555f1902ec6348" candidate_uuid: "daef68b474584e4ba68d5396c0af4a9f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c5c18540913498ebc466583ae18de30"
I20260806 03:50:51.270725 5328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef8fc29bc5c840a0ad555f1902ec6348" candidate_uuid: "daef68b474584e4ba68d5396c0af4a9f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "882521371a464d22b968b8863e2e7ba1"
I20260806 03:50:51.270749 4976 raft_consensus.cc:3037] T ef8fc29bc5c840a0ad555f1902ec6348 P 5c5c18540913498ebc466583ae18de30 [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:50:51.270793 5328 raft_consensus.cc:3037] T ef8fc29bc5c840a0ad555f1902ec6348 P 882521371a464d22b968b8863e2e7ba1 [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:50:51.271450 5328 raft_consensus.cc:2454] T ef8fc29bc5c840a0ad555f1902ec6348 P 882521371a464d22b968b8863e2e7ba1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate daef68b474584e4ba68d5396c0af4a9f in term 1.
I20260806 03:50:51.271512 4976 raft_consensus.cc:2454] T ef8fc29bc5c840a0ad555f1902ec6348 P 5c5c18540913498ebc466583ae18de30 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate daef68b474584e4ba68d5396c0af4a9f in term 1.
I20260806 03:50:51.271598 5070 leader_election.cc:302] T ef8fc29bc5c840a0ad555f1902ec6348 P daef68b474584e4ba68d5396c0af4a9f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 882521371a464d22b968b8863e2e7ba1, daef68b474584e4ba68d5396c0af4a9f; no voters:
I20260806 03:50:51.271683 5411 raft_consensus.cc:2781] T ef8fc29bc5c840a0ad555f1902ec6348 P daef68b474584e4ba68d5396c0af4a9f [term 1 FOLLOWER]: Leader election won for term 1
I20260806 03:50:51.271826 5411 raft_consensus.cc:686] T ef8fc29bc5c840a0ad555f1902ec6348 P daef68b474584e4ba68d5396c0af4a9f [term 1 LEADER]: Becoming Leader. State: Replica: daef68b474584e4ba68d5396c0af4a9f, State: Running, Role: LEADER
I20260806 03:50:51.271934 5411 consensus_queue.cc:237] T ef8fc29bc5c840a0ad555f1902ec6348 P daef68b474584e4ba68d5396c0af4a9f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c5c18540913498ebc466583ae18de30" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 39265 } } peers { permanent_uuid: "882521371a464d22b968b8863e2e7ba1" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 44629 } } peers { permanent_uuid: "daef68b474584e4ba68d5396c0af4a9f" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 37949 } }
I20260806 03:50:51.272807 4791 catalog_manager.cc:5495] T ef8fc29bc5c840a0ad555f1902ec6348 P daef68b474584e4ba68d5396c0af4a9f reported cstate change: term changed from 0 to 1, leader changed from <none> to daef68b474584e4ba68d5396c0af4a9f (127.11.253.194). New cstate: current_term: 1 leader_uuid: "daef68b474584e4ba68d5396c0af4a9f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c5c18540913498ebc466583ae18de30" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 39265 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "882521371a464d22b968b8863e2e7ba1" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 44629 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "daef68b474584e4ba68d5396c0af4a9f" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 37949 } health_report { overall_health: HEALTHY } } }
W20260806 03:50:51.293784 5047 tablet.cc:2369] T ef8fc29bc5c840a0ad555f1902ec6348 P 5c5c18540913498ebc466583ae18de30: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 03:50:51.302909 5387 tablet.cc:2369] T ef8fc29bc5c840a0ad555f1902ec6348 P 882521371a464d22b968b8863e2e7ba1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 03:50:51.311671 5403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:50:51.309260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33024 (local address 127.11.253.194:37949)
0806 03:50:51.309349 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0806 03:50:51.309351 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:50:51.309359 (+ 8us) server_negotiation.cc:407] Connection header received
0806 03:50:51.309405 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:50:51.309406 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:50:51.309429 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:50:51.309493 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 03:50:51.309632 (+ 139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:50:51.310125 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:50:51.310887 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:50:51.311086 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:50:51.311154 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:50:51.311295 (+ 141us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 03:50:51.311401 (+ 106us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 03:50:51.311467 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0806 03:50:51.311573 (+ 106us) server_negotiation.cc:299] Negotiation successful
0806 03:50:51.311591 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260806 03:50:51.314188 5328 raft_consensus.cc:1261] T ef8fc29bc5c840a0ad555f1902ec6348 P 882521371a464d22b968b8863e2e7ba1 [term 1 FOLLOWER]: Refusing update from remote peer daef68b474584e4ba68d5396c0af4a9f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 03:50:51.314185 4976 raft_consensus.cc:1261] T ef8fc29bc5c840a0ad555f1902ec6348 P 5c5c18540913498ebc466583ae18de30 [term 1 FOLLOWER]: Refusing update from remote peer daef68b474584e4ba68d5396c0af4a9f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 03:50:51.314671 5414 consensus_queue.cc:1035] T ef8fc29bc5c840a0ad555f1902ec6348 P daef68b474584e4ba68d5396c0af4a9f [LEADER]: Connected to new peer: Peer: permanent_uuid: "882521371a464d22b968b8863e2e7ba1" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 44629 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 03:50:51.314862 5411 consensus_queue.cc:1035] T ef8fc29bc5c840a0ad555f1902ec6348 P daef68b474584e4ba68d5396c0af4a9f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c5c18540913498ebc466583ae18de30" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 39265 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 03:50:51.317271 5417 mvcc.cc:205] Tried to move back new op lower bound from 7315407877378793472 to 7315407877210419200. Current Snapshot: MvccSnapshot[applied={T|T < 7315407877378793472}]
I20260806 03:50:51.317526 5418 mvcc.cc:205] Tried to move back new op lower bound from 7315407877378793472 to 7315407877210419200. Current Snapshot: MvccSnapshot[applied={T|T < 7315407877378793472}]
I20260806 03:50:51.318121 5419 mvcc.cc:205] Tried to move back new op lower bound from 7315407877378793472 to 7315407877210419200. Current Snapshot: MvccSnapshot[applied={T|T < 7315407877378793472}]
I20260806 03:50:51.320355 4791 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:50458:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 03:50:51.320449 4791 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:50458:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 03:50:51.321254 4791 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 237c89cd5c5d4a50877ab4a0c0e27903: Sending DeleteTablet for 3 replicas of tablet ef8fc29bc5c840a0ad555f1902ec6348
I20260806 03:50:51.321587 5134 tablet_service.cc:1514] Processing DeleteTablet for tablet ef8fc29bc5c840a0ad555f1902ec6348 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 03:50:51 UTC) from {username='slave'} at 127.0.0.1:33018
I20260806 03:50:51.321585 4946 tablet_service.cc:1514] Processing DeleteTablet for tablet ef8fc29bc5c840a0ad555f1902ec6348 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 03:50:51 UTC) from {username='slave'} at 127.0.0.1:36744
I20260806 03:50:51.321974 5302 tablet_service.cc:1514] Processing DeleteTablet for tablet ef8fc29bc5c840a0ad555f1902ec6348 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 03:50:51 UTC) from {username='slave'} at 127.0.0.1:33042
I20260806 03:50:51.322059 12279 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskToVWRe/build/release/bin/kudu with pid 4865
I20260806 03:50:51.327641 5429 tablet_replica.cc:330] T ef8fc29bc5c840a0ad555f1902ec6348 P daef68b474584e4ba68d5396c0af4a9f: stopping tablet replica
I20260806 03:50:51.327805 5429 raft_consensus.cc:2229] T ef8fc29bc5c840a0ad555f1902ec6348 P daef68b474584e4ba68d5396c0af4a9f [term 1 LEADER]: Raft consensus shutting down.
I20260806 03:50:51.328001 5429 raft_consensus.cc:2258] T ef8fc29bc5c840a0ad555f1902ec6348 P daef68b474584e4ba68d5396c0af4a9f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 03:50:51.329941 5429 ts_tablet_manager.cc:1905] T ef8fc29bc5c840a0ad555f1902ec6348 P daef68b474584e4ba68d5396c0af4a9f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 03:50:51.331755 5429 ts_tablet_manager.cc:1918] T ef8fc29bc5c840a0ad555f1902ec6348 P daef68b474584e4ba68d5396c0af4a9f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 03:50:51.331818 5429 log.cc:1196] T ef8fc29bc5c840a0ad555f1902ec6348 P daef68b474584e4ba68d5396c0af4a9f: Deleting WAL directory at /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785988004637995-12279-0/minicluster-data/ts-1/wal/wals/ef8fc29bc5c840a0ad555f1902ec6348
I20260806 03:50:51.332049 5429 ts_tablet_manager.cc:1939] T ef8fc29bc5c840a0ad555f1902ec6348 P daef68b474584e4ba68d5396c0af4a9f: Deleting consensus metadata
I20260806 03:50:51.332450 4773 catalog_manager.cc:4841] TS daef68b474584e4ba68d5396c0af4a9f (127.11.253.194:37949): tablet ef8fc29bc5c840a0ad555f1902ec6348 (table test-table [id=6be08c1105db409a97e41ab39e14c5ab]) successfully deleted
I20260806 03:50:51.333220 5431 tablet_replica.cc:330] T ef8fc29bc5c840a0ad555f1902ec6348 P 882521371a464d22b968b8863e2e7ba1: stopping tablet replica
I20260806 03:50:51.333324 5431 raft_consensus.cc:2229] T ef8fc29bc5c840a0ad555f1902ec6348 P 882521371a464d22b968b8863e2e7ba1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 03:50:51.333410 5431 raft_consensus.cc:2258] T ef8fc29bc5c840a0ad555f1902ec6348 P 882521371a464d22b968b8863e2e7ba1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 03:50:51.333971 5431 ts_tablet_manager.cc:1905] T ef8fc29bc5c840a0ad555f1902ec6348 P 882521371a464d22b968b8863e2e7ba1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 03:50:51.341099 5431 ts_tablet_manager.cc:1918] T ef8fc29bc5c840a0ad555f1902ec6348 P 882521371a464d22b968b8863e2e7ba1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 03:50:51.341212 5431 log.cc:1196] T ef8fc29bc5c840a0ad555f1902ec6348 P 882521371a464d22b968b8863e2e7ba1: Deleting WAL directory at /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785988004637995-12279-0/minicluster-data/ts-2/wal/wals/ef8fc29bc5c840a0ad555f1902ec6348
I20260806 03:50:51.341495 5431 ts_tablet_manager.cc:1939] T ef8fc29bc5c840a0ad555f1902ec6348 P 882521371a464d22b968b8863e2e7ba1: Deleting consensus metadata
I20260806 03:50:51.342036 4772 catalog_manager.cc:4841] TS 882521371a464d22b968b8863e2e7ba1 (127.11.253.195:44629): tablet ef8fc29bc5c840a0ad555f1902ec6348 (table test-table [id=6be08c1105db409a97e41ab39e14c5ab]) successfully deleted
W20260806 03:50:51.345387 4770 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.253.193:39265 (error 108)
I20260806 03:50:51.346036 5398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:50:51.345842 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.193:39265 (local address 127.0.0.1:36756)
0806 03:50:51.345942 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0806 03:50:51.345997 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.253.193:39265: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260806 03:50:51.346150 12279 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskToVWRe/build/release/bin/kudu with pid 5049
W20260806 03:50:51.354395 4770 catalog_manager.cc:4584] TS 5c5c18540913498ebc466583ae18de30 (127.11.253.193:39265): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet ef8fc29bc5c840a0ad555f1902ec6348: Network error: Client connection negotiation failed: client connection to 127.11.253.193:39265: connect: Connection refused (error 111)
I20260806 03:50:51.359820 12279 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskToVWRe/build/release/bin/kudu with pid 5223
I20260806 03:50:51.367703 12279 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskToVWRe/build/release/bin/kudu with pid 4751
2026-08-06T03:50:51Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (896 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785988004637995-12279-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 03:50:51 dist-test-slave-6ntk krb5kdc[5436](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 03:50:51 dist-test-slave-6ntk krb5kdc[5436](info): set up 2 sockets
Aug 06 03:50:51 dist-test-slave-6ntk krb5kdc[5436](info): commencing operation
krb5kdc: starting...
W20260806 03:50:52.618793 12279 mini_kdc.cc:121] Time spent starting KDC: real 1.224s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785988004637995-12279-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785988004637995-12279-0/krb5kdc/test-user.keytab.
Aug 06 03:50:52 dist-test-slave-6ntk krb5kdc[5436](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988252, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T03:50:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T03:50:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.253.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.253.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.253.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.254.keytab.
Entry for principal kudu/127.11.253.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.254.keytab.
I20260806 03:50:52.761404 12279 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskToVWRe/build/release/bin/kudu
/tmp/dist-test-taskToVWRe/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785988004637995-12279-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785988004637995-12279-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785988004637995-12279-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785988004637995-12279-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.253.254:37975
--webserver_interface=127.11.253.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.254.keytab
--principal=kudu/127.11.253.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.253.212:43659
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.253.254:37975
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785988004637995-12279-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785988004637995-12279-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785988004637995-12279-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:50:52.845477 5455 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:50:52.845739 5455 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:50:52.845777 5455 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:50:52.847002 5455 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 03:50:52.847080 5455 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:50:52.847105 5455 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:50:52.847127 5455 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 03:50:52.847155 5455 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 03:50:52.848690 5455 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.253.212:43659
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785988004637995-12279-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785988004637995-12279-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.253.254:37975
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.254.keytab
--principal=kudu/127.11.253.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.253.254:37975
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785988004637995-12279-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.253.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5455
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785988004637995-12279-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:50:52.848976 5455 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:50:52.849218 5455 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 03:50:52.854364 5455 server_base.cc:1034] running on GCE node
W20260806 03:50:52.854274 5465 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:50:52.854290 5463 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:50:52.854282 5462 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:50:52.854769 5455 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:50:52.855093 5455 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:50:52.856290 5455 hybrid_clock.cc:648] HybridClock initialized: now 1785988252856274 us; error 28 us; skew 500 ppm
Aug 06 03:50:52 dist-test-slave-6ntk krb5kdc[5436](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988252, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.253.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:50:52.859014 5455 init.cc:377] Logged in from keytab as kudu/127.11.253.254@KRBTEST.COM (short username kudu)
I20260806 03:50:52.859431 5455 webserver.cc:460] Webserver started at http://127.11.253.254:40569/ using document root <none> and password file <none>
I20260806 03:50:52.859637 5455 fs_manager.cc:359] Metadata directory not provided
I20260806 03:50:52.859676 5455 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:50:52.859802 5455 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:50:52.860828 5455 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785988004637995-12279-0/minicluster-data/master-0/data/instance:
uuid: "de8cab405fea49fbad8597dfce1c9827"
format_stamp: "Formatted at 2026-08-06 03:50:52 on dist-test-slave-6ntk"
server_key: "980dde12c59a14b110ce3ff3014ca74b"
server_key_iv: "f764b2415b6f064573795881b1231257"
server_key_version: "encryptionkey@0"
I20260806 03:50:52.861129 5455 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785988004637995-12279-0/minicluster-data/master-0/wal/instance:
uuid: "de8cab405fea49fbad8597dfce1c9827"
format_stamp: "Formatted at 2026-08-06 03:50:52 on dist-test-slave-6ntk"
server_key: "980dde12c59a14b110ce3ff3014ca74b"
server_key_iv: "f764b2415b6f064573795881b1231257"
server_key_version: "encryptionkey@0"
I20260806 03:50:52.862515 5455 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 03:50:52.863214 5473 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:50:52.863457 5455 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 03:50:52.863545 5455 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785988004637995-12279-0/minicluster-data/master-0/data,/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785988004637995-12279-0/minicluster-data/master-0/wal
uuid: "de8cab405fea49fbad8597dfce1c9827"
format_stamp: "Formatted at 2026-08-06 03:50:52 on dist-test-slave-6ntk"
server_key: "980dde12c59a14b110ce3ff3014ca74b"
server_key_iv: "f764b2415b6f064573795881b1231257"
server_key_version: "encryptionkey@0"
I20260806 03:50:52.863602 5455 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785988004637995-12279-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785988004637995-12279-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785988004637995-12279-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:50:52.877558 5455 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:50:52.879477 5455 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:50:52.879611 5455 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:50:52.883749 5455 rpc_server.cc:307] RPC server started. Bound to: 127.11.253.254:37975
I20260806 03:50:52.883806 5543 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.253.254:37975 every 8 connection(s)
I20260806 03:50:52.884146 5455 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785988004637995-12279-0/minicluster-data/master-0/data/info.pb
I20260806 03:50:52.885115 5544 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:50:52.887006 12279 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskToVWRe/build/release/bin/kudu as pid 5455
I20260806 03:50:52.887123 12279 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785988004637995-12279-0/minicluster-data/master-0/wal/instance
I20260806 03:50:52.887380 12279 external_mini_cluster.cc:1428] Setting key b227f438efb03e9b3ae415d92b668d61
I20260806 03:50:52.887732 5544 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P de8cab405fea49fbad8597dfce1c9827: Bootstrap starting.
I20260806 03:50:52.888438 5544 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P de8cab405fea49fbad8597dfce1c9827: Neither blocks nor log segments found. Creating new log.
I20260806 03:50:52.888784 5544 log.cc:824] T 00000000000000000000000000000000 P de8cab405fea49fbad8597dfce1c9827: Log is configured to *not* fsync() on all Append() calls
I20260806 03:50:52.889667 5544 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P de8cab405fea49fbad8597dfce1c9827: No bootstrap required, opened a new log
I20260806 03:50:52.892088 5544 raft_consensus.cc:348] T 00000000000000000000000000000000 P de8cab405fea49fbad8597dfce1c9827 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de8cab405fea49fbad8597dfce1c9827" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 37975 } }
I20260806 03:50:52.892279 5544 raft_consensus.cc:374] T 00000000000000000000000000000000 P de8cab405fea49fbad8597dfce1c9827 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:50:52.892316 5544 raft_consensus.cc:729] T 00000000000000000000000000000000 P de8cab405fea49fbad8597dfce1c9827 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de8cab405fea49fbad8597dfce1c9827, State: Initialized, Role: FOLLOWER
I20260806 03:50:52.892483 5544 consensus_queue.cc:260] T 00000000000000000000000000000000 P de8cab405fea49fbad8597dfce1c9827 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de8cab405fea49fbad8597dfce1c9827" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 37975 } }
I20260806 03:50:52.892594 5544 raft_consensus.cc:388] T 00000000000000000000000000000000 P de8cab405fea49fbad8597dfce1c9827 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 03:50:52.892693 5544 raft_consensus.cc:482] T 00000000000000000000000000000000 P de8cab405fea49fbad8597dfce1c9827 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 03:50:52.892745 5544 raft_consensus.cc:3037] T 00000000000000000000000000000000 P de8cab405fea49fbad8597dfce1c9827 [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:50:52.893610 5544 raft_consensus.cc:504] T 00000000000000000000000000000000 P de8cab405fea49fbad8597dfce1c9827 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de8cab405fea49fbad8597dfce1c9827" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 37975 } }
I20260806 03:50:52.893741 5544 leader_election.cc:302] T 00000000000000000000000000000000 P de8cab405fea49fbad8597dfce1c9827 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: de8cab405fea49fbad8597dfce1c9827; no voters:
I20260806 03:50:52.893953 5544 leader_election.cc:288] T 00000000000000000000000000000000 P de8cab405fea49fbad8597dfce1c9827 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 03:50:52.893987 5552 raft_consensus.cc:2781] T 00000000000000000000000000000000 P de8cab405fea49fbad8597dfce1c9827 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 03:50:52.894169 5552 raft_consensus.cc:686] T 00000000000000000000000000000000 P de8cab405fea49fbad8597dfce1c9827 [term 1 LEADER]: Becoming Leader. State: Replica: de8cab405fea49fbad8597dfce1c9827, State: Running, Role: LEADER
I20260806 03:50:52.894258 5544 sys_catalog.cc:564] T 00000000000000000000000000000000 P de8cab405fea49fbad8597dfce1c9827 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 03:50:52.894336 5552 consensus_queue.cc:237] T 00000000000000000000000000000000 P de8cab405fea49fbad8597dfce1c9827 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de8cab405fea49fbad8597dfce1c9827" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 37975 } }
I20260806 03:50:52.894640 5552 sys_catalog.cc:455] T 00000000000000000000000000000000 P de8cab405fea49fbad8597dfce1c9827 [sys.catalog]: SysCatalogTable state changed. Reason: New leader de8cab405fea49fbad8597dfce1c9827. Latest consensus state: current_term: 1 leader_uuid: "de8cab405fea49fbad8597dfce1c9827" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de8cab405fea49fbad8597dfce1c9827" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 37975 } } }
I20260806 03:50:52.894729 5552 sys_catalog.cc:458] T 00000000000000000000000000000000 P de8cab405fea49fbad8597dfce1c9827 [sys.catalog]: This master's current role is: LEADER
I20260806 03:50:52.894651 5554 sys_catalog.cc:455] T 00000000000000000000000000000000 P de8cab405fea49fbad8597dfce1c9827 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "de8cab405fea49fbad8597dfce1c9827" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de8cab405fea49fbad8597dfce1c9827" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 37975 } } }
I20260806 03:50:52.894894 5554 sys_catalog.cc:458] T 00000000000000000000000000000000 P de8cab405fea49fbad8597dfce1c9827 [sys.catalog]: This master's current role is: LEADER
Aug 06 03:50:52 dist-test-slave-6ntk krb5kdc[5436](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988252, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.253.254@KRBTEST.COM
W20260806 03:50:52.896281 5573 catalog_manager.cc:1558] T 00000000000000000000000000000000 P de8cab405fea49fbad8597dfce1c9827: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 03:50:52.896377 5573 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 03:50:52.896739 5559 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 03:50:52.896971 5559 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 03:50:52.898743 5559 catalog_manager.cc:1347] Generated new cluster ID: 9a92809e8fe14f1b9aeb30de07ccc234
I20260806 03:50:52.898797 5559 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 03:50:52.900199 5550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:50:52.887964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48452 (local address 127.11.253.254:37975)
0806 03:50:52.888163 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0806 03:50:52.888171 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 03:50:52.888207 (+ 36us) server_negotiation.cc:407] Connection header received
0806 03:50:52.888780 (+ 573us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:50:52.888788 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:50:52.888840 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:50:52.889024 (+ 184us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:50:52.889829 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:50:52.890870 (+ 1041us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:50:52.892146 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:50:52.892891 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:50:52.892996 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:50:52.896311 (+ 3315us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:50:52.896315 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:50:52.896316 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:50:52.896328 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:50:52.898449 (+ 2121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:50:52.898891 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:50:52.898894 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:50:52.898894 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:50:52.898997 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:50:52.899188 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:50:52.899189 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:50:52.899189 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:50:52.899467 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:50:52.899588 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0806 03:50:52.899848 (+ 260us) server_negotiation.cc:299] Negotiation successful
0806 03:50:52.899984 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":102,"threads_started":1}
I20260806 03:50:52.903137 5559 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 03:50:52.903841 5559 catalog_manager.cc:1504] Loading token signing keys...
I20260806 03:50:52.908396 5559 catalog_manager.cc:5868] T 00000000000000000000000000000000 P de8cab405fea49fbad8597dfce1c9827: Generated new TSK 0
I20260806 03:50:52.908562 5559 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.253.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.253.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.253.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.193.keytab.
Entry for principal kudu/127.11.253.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.193.keytab.
I20260806 03:50:52.937744 12279 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskToVWRe/build/release/bin/kudu
/tmp/dist-test-taskToVWRe/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785988004637995-12279-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785988004637995-12279-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785988004637995-12279-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785988004637995-12279-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.253.193:0
--local_ip_for_outbound_sockets=127.11.253.193
--webserver_interface=127.11.253.193
--webserver_port=0
--tserver_master_addrs=127.11.253.254:37975
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.193.keytab
--principal=kudu/127.11.253.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.253.212:43659
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785988004637995-12279-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785988004637995-12279-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785988004637995-12279-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:50:53.028679 5579 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:50:53.028944 5579 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:50:53.028983 5579 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:50:53.030191 5579 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:50:53.030263 5579 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:50:53.030308 5579 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.253.193
I20260806 03:50:53.031857 5579 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.253.212:43659
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785988004637995-12279-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785988004637995-12279-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.193.keytab
--principal=kudu/127.11.253.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.253.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785988004637995-12279-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.253.193
--webserver_port=0
--tserver_master_addrs=127.11.253.254:37975
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5579
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.253.193
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785988004637995-12279-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:50:53.032142 5579 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:50:53.032397 5579 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:50:53.037576 5593 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:50:53.037676 5588 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:50:53.037580 5587 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:50:53.038039 5579 server_base.cc:1034] running on GCE node
I20260806 03:50:53.038259 5579 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:50:53.038504 5579 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:50:53.039680 5579 hybrid_clock.cc:648] HybridClock initialized: now 1785988253039658 us; error 35 us; skew 500 ppm
Aug 06 03:50:53 dist-test-slave-6ntk krb5kdc[5436](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988253, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.253.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:50:53.042680 5579 init.cc:377] Logged in from keytab as kudu/127.11.253.193@KRBTEST.COM (short username kudu)
I20260806 03:50:53.043134 5579 webserver.cc:460] Webserver started at http://127.11.253.193:40915/ using document root <none> and password file <none>
I20260806 03:50:53.043388 5579 fs_manager.cc:359] Metadata directory not provided
I20260806 03:50:53.043459 5579 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:50:53.043608 5579 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:50:53.044691 5579 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785988004637995-12279-0/minicluster-data/ts-0/data/instance:
uuid: "89a6c7b7d5824849ba9579fdf6ab9b8c"
format_stamp: "Formatted at 2026-08-06 03:50:53 on dist-test-slave-6ntk"
server_key: "b302413611d080cb3dd0eeb8d02af77c"
server_key_iv: "3d2fd2b7201fd00b29802ca37ef0aa87"
server_key_version: "encryptionkey@0"
I20260806 03:50:53.045066 5579 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785988004637995-12279-0/minicluster-data/ts-0/wal/instance:
uuid: "89a6c7b7d5824849ba9579fdf6ab9b8c"
format_stamp: "Formatted at 2026-08-06 03:50:53 on dist-test-slave-6ntk"
server_key: "b302413611d080cb3dd0eeb8d02af77c"
server_key_iv: "3d2fd2b7201fd00b29802ca37ef0aa87"
server_key_version: "encryptionkey@0"
I20260806 03:50:53.046556 5579 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 03:50:53.047524 5604 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:50:53.047726 5579 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 03:50:53.047858 5579 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785988004637995-12279-0/minicluster-data/ts-0/data,/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785988004637995-12279-0/minicluster-data/ts-0/wal
uuid: "89a6c7b7d5824849ba9579fdf6ab9b8c"
format_stamp: "Formatted at 2026-08-06 03:50:53 on dist-test-slave-6ntk"
server_key: "b302413611d080cb3dd0eeb8d02af77c"
server_key_iv: "3d2fd2b7201fd00b29802ca37ef0aa87"
server_key_version: "encryptionkey@0"
I20260806 03:50:53.047946 5579 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785988004637995-12279-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785988004637995-12279-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785988004637995-12279-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:50:53.056871 5579 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:50:53.059096 5579 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:50:53.059290 5579 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:50:53.059556 5579 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 03:50:53.059926 5579 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 03:50:53.059989 5579 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 03:50:53.060040 5579 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 03:50:53.060070 5579 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 03:50:53.066958 5579 rpc_server.cc:307] RPC server started. Bound to: 127.11.253.193:45311
I20260806 03:50:53.067000 5745 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.253.193:45311 every 8 connection(s)
I20260806 03:50:53.067430 5579 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785988004637995-12279-0/minicluster-data/ts-0/data/info.pb
Aug 06 03:50:53 dist-test-slave-6ntk krb5kdc[5436](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988253, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.253.193@KRBTEST.COM for kudu/127.11.253.254@KRBTEST.COM
I20260806 03:50:53.075114 12279 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskToVWRe/build/release/bin/kudu as pid 5579
I20260806 03:50:53.075238 12279 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785988004637995-12279-0/minicluster-data/ts-0/wal/instance
I20260806 03:50:53.075462 12279 external_mini_cluster.cc:1428] Setting key 99286b1c3bfaaae117fac492fa00dd56
I20260806 03:50:53.077709 5749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:50:53.068141 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:37975 (local address 127.11.253.193:39061)
0806 03:50:53.068345 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0806 03:50:53.068407 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0806 03:50:53.068942 (+ 535us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:50:53.069152 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:50:53.069156 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:50:53.069794 (+ 638us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:50:53.070028 (+ 234us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:50:53.070033 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:50:53.070743 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:50:53.070744 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:50:53.071602 (+ 858us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:50:53.071605 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:50:53.072014 (+ 409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:50:53.072015 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:50:53.072146 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:50:53.072682 (+ 536us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:50:53.072691 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:50:53.074254 (+ 1563us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:50:53.076258 (+ 2004us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:50:53.076267 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:50:53.076270 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:50:53.076514 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:50:53.076763 (+ 249us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:50:53.076763 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:50:53.076764 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:50:53.076884 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:50:53.077198 (+ 314us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:50:53.077203 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:50:53.077326 (+ 123us) client_negotiation.cc:769] Sending connection context
0806 03:50:53.077412 (+ 86us) client_negotiation.cc:240] Negotiation successful
0806 03:50:53.077536 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":103,"threads_started":1}
I20260806 03:50:53.077741 5550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:50:53.068186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.253.193:39061 (local address 127.11.253.254:37975)
0806 03:50:53.068269 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0806 03:50:53.068271 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:50:53.068515 (+ 244us) server_negotiation.cc:407] Connection header received
0806 03:50:53.069032 (+ 517us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:50:53.069032 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:50:53.069043 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:50:53.069095 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:50:53.070120 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:50:53.070637 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:50:53.071696 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:50:53.071908 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:50:53.071967 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:50:53.074355 (+ 2388us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:50:53.074356 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:50:53.074356 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:50:53.074364 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:50:53.076122 (+ 1758us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:50:53.076608 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:50:53.076609 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:50:53.076610 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:50:53.076685 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:50:53.076956 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:50:53.076957 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:50:53.076958 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:50:53.077127 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:50:53.077217 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0806 03:50:53.077465 (+ 248us) server_negotiation.cc:299] Negotiation successful
0806 03:50:53.077590 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260806 03:50:53.078315 5747 heartbeater.cc:347] Connected to a master server at 127.11.253.254:37975
I20260806 03:50:53.078447 5747 heartbeater.cc:464] Registering TS with master...
I20260806 03:50:53.078660 5747 heartbeater.cc:511] Master 127.11.253.254:37975 requested a full tablet report, sending...
I20260806 03:50:53.079283 5499 ts_manager.cc:194] Registered new tserver with Master: 89a6c7b7d5824849ba9579fdf6ab9b8c (127.11.253.193:45311)
I20260806 03:50:53.080089 5499 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.253.193@KRBTEST.COM'} at 127.11.253.193:39061
WARNING: no policy specified for kudu/127.11.253.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.253.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.253.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.194.keytab.
Entry for principal kudu/127.11.253.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.194.keytab.
I20260806 03:50:53.105151 12279 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskToVWRe/build/release/bin/kudu
/tmp/dist-test-taskToVWRe/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785988004637995-12279-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785988004637995-12279-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785988004637995-12279-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785988004637995-12279-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.253.194:0
--local_ip_for_outbound_sockets=127.11.253.194
--webserver_interface=127.11.253.194
--webserver_port=0
--tserver_master_addrs=127.11.253.254:37975
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.194.keytab
--principal=kudu/127.11.253.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.253.212:43659
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785988004637995-12279-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785988004637995-12279-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785988004637995-12279-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:50:53.191480 5754 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:50:53.191756 5754 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:50:53.191797 5754 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:50:53.192986 5754 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:50:53.193068 5754 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:50:53.193113 5754 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.253.194
I20260806 03:50:53.194846 5754 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.253.212:43659
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785988004637995-12279-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785988004637995-12279-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.194.keytab
--principal=kudu/127.11.253.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.253.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785988004637995-12279-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.253.194
--webserver_port=0
--tserver_master_addrs=127.11.253.254:37975
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5754
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.253.194
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785988004637995-12279-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:50:53.195163 5754 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:50:53.195451 5754 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:50:53.201025 5760 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:50:53.201107 5759 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:50:53.201030 5762 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:50:53.201812 5754 server_base.cc:1034] running on GCE node
I20260806 03:50:53.202113 5754 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:50:53.202396 5754 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:50:53.203603 5754 hybrid_clock.cc:648] HybridClock initialized: now 1785988253203597 us; error 41 us; skew 500 ppm
Aug 06 03:50:53 dist-test-slave-6ntk krb5kdc[5436](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988253, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.253.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:50:53.206471 5754 init.cc:377] Logged in from keytab as kudu/127.11.253.194@KRBTEST.COM (short username kudu)
I20260806 03:50:53.206954 5754 webserver.cc:460] Webserver started at http://127.11.253.194:33859/ using document root <none> and password file <none>
I20260806 03:50:53.207223 5754 fs_manager.cc:359] Metadata directory not provided
I20260806 03:50:53.207279 5754 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:50:53.207425 5754 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:50:53.208621 5754 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785988004637995-12279-0/minicluster-data/ts-1/data/instance:
uuid: "b632a244bf9c4689ad518228e3bf96d4"
format_stamp: "Formatted at 2026-08-06 03:50:53 on dist-test-slave-6ntk"
server_key: "9989c45f0c75fc6f1c3066da222530e6"
server_key_iv: "e54f0586371558ae94aa7ad44ad4d0fc"
server_key_version: "encryptionkey@0"
I20260806 03:50:53.208992 5754 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785988004637995-12279-0/minicluster-data/ts-1/wal/instance:
uuid: "b632a244bf9c4689ad518228e3bf96d4"
format_stamp: "Formatted at 2026-08-06 03:50:53 on dist-test-slave-6ntk"
server_key: "9989c45f0c75fc6f1c3066da222530e6"
server_key_iv: "e54f0586371558ae94aa7ad44ad4d0fc"
server_key_version: "encryptionkey@0"
I20260806 03:50:53.210567 5754 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 03:50:53.211511 5770 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:50:53.211913 5754 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 03:50:53.212025 5754 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785988004637995-12279-0/minicluster-data/ts-1/data,/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785988004637995-12279-0/minicluster-data/ts-1/wal
uuid: "b632a244bf9c4689ad518228e3bf96d4"
format_stamp: "Formatted at 2026-08-06 03:50:53 on dist-test-slave-6ntk"
server_key: "9989c45f0c75fc6f1c3066da222530e6"
server_key_iv: "e54f0586371558ae94aa7ad44ad4d0fc"
server_key_version: "encryptionkey@0"
I20260806 03:50:53.212109 5754 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785988004637995-12279-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785988004637995-12279-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785988004637995-12279-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:50:53.225792 5754 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:50:53.227980 5754 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:50:53.228143 5754 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:50:53.228401 5754 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 03:50:53.228740 5754 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 03:50:53.228799 5754 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 03:50:53.228825 5754 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 03:50:53.228838 5754 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 03:50:53.234965 5754 rpc_server.cc:307] RPC server started. Bound to: 127.11.253.194:38787
I20260806 03:50:53.235097 5919 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.253.194:38787 every 8 connection(s)
I20260806 03:50:53.235388 5754 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785988004637995-12279-0/minicluster-data/ts-1/data/info.pb
I20260806 03:50:53.241406 12279 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskToVWRe/build/release/bin/kudu as pid 5754
I20260806 03:50:53.241586 12279 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785988004637995-12279-0/minicluster-data/ts-1/wal/instance
I20260806 03:50:53.241837 12279 external_mini_cluster.cc:1428] Setting key b3a3ee75265fd645361a4cf0080f1acc
Aug 06 03:50:53 dist-test-slave-6ntk krb5kdc[5436](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988253, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.253.194@KRBTEST.COM for kudu/127.11.253.254@KRBTEST.COM
I20260806 03:50:53.245836 5550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:50:53.236222 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.253.194:36915 (local address 127.11.253.254:37975)
0806 03:50:53.236324 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0806 03:50:53.236326 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:50:53.236494 (+ 168us) server_negotiation.cc:407] Connection header received
0806 03:50:53.236974 (+ 480us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:50:53.236975 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:50:53.236984 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:50:53.237059 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:50:53.238073 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:50:53.238640 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:50:53.239718 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:50:53.239924 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:50:53.239984 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:50:53.242512 (+ 2528us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:50:53.242513 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:50:53.242514 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:50:53.242521 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:50:53.244293 (+ 1772us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:50:53.244740 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:50:53.244741 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:50:53.244742 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:50:53.244816 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:50:53.245113 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:50:53.245114 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:50:53.245114 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:50:53.245225 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:50:53.245285 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0806 03:50:53.245654 (+ 369us) server_negotiation.cc:299] Negotiation successful
0806 03:50:53.245764 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260806 03:50:53.246181 5925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:50:53.236190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:37975 (local address 127.11.253.194:36915)
0806 03:50:53.236360 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0806 03:50:53.236402 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 03:50:53.236887 (+ 485us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:50:53.237109 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:50:53.237112 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:50:53.237682 (+ 570us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:50:53.237959 (+ 277us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:50:53.237966 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:50:53.238776 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:50:53.238777 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:50:53.239620 (+ 843us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:50:53.239622 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:50:53.240006 (+ 384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:50:53.240007 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:50:53.240114 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:50:53.240646 (+ 532us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:50:53.240654 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:50:53.242393 (+ 1739us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:50:53.244381 (+ 1988us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:50:53.244388 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:50:53.244390 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:50:53.244645 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:50:53.244892 (+ 247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:50:53.244893 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:50:53.244893 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:50:53.245045 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:50:53.245299 (+ 254us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:50:53.245305 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:50:53.245479 (+ 174us) client_negotiation.cc:769] Sending connection context
0806 03:50:53.245605 (+ 126us) client_negotiation.cc:240] Negotiation successful
0806 03:50:53.245807 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":94,"threads_started":1}
I20260806 03:50:53.246752 5920 heartbeater.cc:347] Connected to a master server at 127.11.253.254:37975
I20260806 03:50:53.246867 5920 heartbeater.cc:464] Registering TS with master...
I20260806 03:50:53.247052 5920 heartbeater.cc:511] Master 127.11.253.254:37975 requested a full tablet report, sending...
I20260806 03:50:53.247488 5499 ts_manager.cc:194] Registered new tserver with Master: b632a244bf9c4689ad518228e3bf96d4 (127.11.253.194:38787)
I20260806 03:50:53.248098 5499 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.253.194@KRBTEST.COM'} at 127.11.253.194:36915
WARNING: no policy specified for kudu/127.11.253.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.253.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.253.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.195.keytab.
Entry for principal kudu/127.11.253.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.195.keytab.
I20260806 03:50:53.274196 12279 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskToVWRe/build/release/bin/kudu
/tmp/dist-test-taskToVWRe/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785988004637995-12279-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785988004637995-12279-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785988004637995-12279-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785988004637995-12279-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.253.195:0
--local_ip_for_outbound_sockets=127.11.253.195
--webserver_interface=127.11.253.195
--webserver_port=0
--tserver_master_addrs=127.11.253.254:37975
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.195.keytab
--principal=kudu/127.11.253.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.253.212:43659
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785988004637995-12279-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785988004637995-12279-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785988004637995-12279-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:50:53.360994 5933 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:50:53.361256 5933 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:50:53.361297 5933 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:50:53.362543 5933 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:50:53.362625 5933 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:50:53.362668 5933 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.253.195
I20260806 03:50:53.364202 5933 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.253.212:43659
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785988004637995-12279-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785988004637995-12279-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.195.keytab
--principal=kudu/127.11.253.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.253.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785988004637995-12279-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.253.195
--webserver_port=0
--tserver_master_addrs=127.11.253.254:37975
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5933
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.253.195
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785988004637995-12279-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:50:53.364506 5933 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:50:53.364768 5933 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:50:53.370283 5939 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:50:53.370316 5942 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:50:53.370347 5938 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:50:53.370476 5933 server_base.cc:1034] running on GCE node
I20260806 03:50:53.370729 5933 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:50:53.371009 5933 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:50:53.372181 5933 hybrid_clock.cc:648] HybridClock initialized: now 1785988253372167 us; error 28 us; skew 500 ppm
Aug 06 03:50:53 dist-test-slave-6ntk krb5kdc[5436](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988253, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.253.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:50:53.380478 5933 init.cc:377] Logged in from keytab as kudu/127.11.253.195@KRBTEST.COM (short username kudu)
I20260806 03:50:53.380949 5933 webserver.cc:460] Webserver started at http://127.11.253.195:43679/ using document root <none> and password file <none>
I20260806 03:50:53.381188 5933 fs_manager.cc:359] Metadata directory not provided
I20260806 03:50:53.381316 5933 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:50:53.381472 5933 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:50:53.382658 5933 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785988004637995-12279-0/minicluster-data/ts-2/data/instance:
uuid: "06093a7b77f5416c903d79d15353a38c"
format_stamp: "Formatted at 2026-08-06 03:50:53 on dist-test-slave-6ntk"
server_key: "431183b04643d93746367a72103c8d24"
server_key_iv: "a3e92e6114bd37d2ce6adcf4ac3c5746"
server_key_version: "encryptionkey@0"
I20260806 03:50:53.383040 5933 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785988004637995-12279-0/minicluster-data/ts-2/wal/instance:
uuid: "06093a7b77f5416c903d79d15353a38c"
format_stamp: "Formatted at 2026-08-06 03:50:53 on dist-test-slave-6ntk"
server_key: "431183b04643d93746367a72103c8d24"
server_key_iv: "a3e92e6114bd37d2ce6adcf4ac3c5746"
server_key_version: "encryptionkey@0"
I20260806 03:50:53.384419 5933 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 03:50:53.385171 5953 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:50:53.385378 5933 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 03:50:53.385492 5933 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785988004637995-12279-0/minicluster-data/ts-2/data,/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785988004637995-12279-0/minicluster-data/ts-2/wal
uuid: "06093a7b77f5416c903d79d15353a38c"
format_stamp: "Formatted at 2026-08-06 03:50:53 on dist-test-slave-6ntk"
server_key: "431183b04643d93746367a72103c8d24"
server_key_iv: "a3e92e6114bd37d2ce6adcf4ac3c5746"
server_key_version: "encryptionkey@0"
I20260806 03:50:53.385568 5933 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785988004637995-12279-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785988004637995-12279-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785988004637995-12279-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:50:53.403450 5933 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:50:53.405493 5933 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:50:53.405709 5933 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:50:53.406042 5933 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 03:50:53.406399 5933 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 03:50:53.406455 5933 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 03:50:53.406500 5933 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 03:50:53.406531 5933 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 03:50:53.413218 5933 rpc_server.cc:307] RPC server started. Bound to: 127.11.253.195:44125
I20260806 03:50:53.413262 6106 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.253.195:44125 every 8 connection(s)
I20260806 03:50:53.413604 5933 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785988004637995-12279-0/minicluster-data/ts-2/data/info.pb
Aug 06 03:50:53 dist-test-slave-6ntk krb5kdc[5436](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988253, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.253.195@KRBTEST.COM for kudu/127.11.253.254@KRBTEST.COM
I20260806 03:50:53.420594 12279 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskToVWRe/build/release/bin/kudu as pid 5933
I20260806 03:50:53.420703 12279 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785988004637995-12279-0/minicluster-data/ts-2/wal/instance
I20260806 03:50:53.421025 12279 external_mini_cluster.cc:1428] Setting key 693ba99a6c69f31d6c1c50583a16a70e
I20260806 03:50:53.423408 5550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:50:53.414340 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.253.195:44881 (local address 127.11.253.254:37975)
0806 03:50:53.414426 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0806 03:50:53.414429 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 03:50:53.414628 (+ 199us) server_negotiation.cc:407] Connection header received
0806 03:50:53.415143 (+ 515us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:50:53.415144 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:50:53.415158 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:50:53.415220 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:50:53.416279 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:50:53.416798 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:50:53.417802 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:50:53.418085 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:50:53.418155 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:50:53.420438 (+ 2283us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:50:53.420440 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:50:53.420440 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:50:53.420449 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:50:53.422179 (+ 1730us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:50:53.422565 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:50:53.422566 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:50:53.422566 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:50:53.422620 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:50:53.422840 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:50:53.422840 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:50:53.422841 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:50:53.422944 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:50:53.422996 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0806 03:50:53.423279 (+ 283us) server_negotiation.cc:299] Negotiation successful
0806 03:50:53.423343 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260806 03:50:53.423549 6111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:50:53.414305 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:37975 (local address 127.11.253.195:44881)
0806 03:50:53.414502 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0806 03:50:53.414540 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0806 03:50:53.415022 (+ 482us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:50:53.415293 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:50:53.415296 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:50:53.415889 (+ 593us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:50:53.416089 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:50:53.416093 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:50:53.416893 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:50:53.416894 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:50:53.417708 (+ 814us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:50:53.417710 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:50:53.418169 (+ 459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:50:53.418170 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:50:53.418278 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:50:53.418878 (+ 600us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:50:53.418887 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:50:53.420318 (+ 1431us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:50:53.422249 (+ 1931us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:50:53.422255 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:50:53.422257 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:50:53.422488 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:50:53.422680 (+ 192us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:50:53.422681 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:50:53.422681 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:50:53.422782 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:50:53.423001 (+ 219us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:50:53.423005 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:50:53.423121 (+ 116us) client_negotiation.cc:769] Sending connection context
0806 03:50:53.423205 (+ 84us) client_negotiation.cc:240] Negotiation successful
0806 03:50:53.423326 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":106,"threads_started":1}
I20260806 03:50:53.424059 6108 heartbeater.cc:347] Connected to a master server at 127.11.253.254:37975
I20260806 03:50:53.424144 6108 heartbeater.cc:464] Registering TS with master...
I20260806 03:50:53.424296 6108 heartbeater.cc:511] Master 127.11.253.254:37975 requested a full tablet report, sending...
I20260806 03:50:53.424640 5499 ts_manager.cc:194] Registered new tserver with Master: 06093a7b77f5416c903d79d15353a38c (127.11.253.195:44125)
I20260806 03:50:53.425029 5499 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.253.195@KRBTEST.COM'} at 127.11.253.195:44881
I20260806 03:50:53.432914 12279 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 03:50:53.440831 5550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:50:53.434400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48458 (local address 127.11.253.254:37975)
0806 03:50:53.434466 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0806 03:50:53.434468 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:50:53.434574 (+ 106us) server_negotiation.cc:407] Connection header received
0806 03:50:53.434613 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:50:53.434613 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:50:53.434622 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:50:53.434694 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:50:53.435214 (+ 520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:50:53.435733 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:50:53.436453 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:50:53.436637 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:50:53.436690 (+ 53us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:50:53.438297 (+ 1607us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:50:53.438298 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:50:53.438298 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:50:53.438305 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:50:53.439684 (+ 1379us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:50:53.440099 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:50:53.440100 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:50:53.440101 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:50:53.440160 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:50:53.440359 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:50:53.440359 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:50:53.440360 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:50:53.440501 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:50:53.440594 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0806 03:50:53.440682 (+ 88us) server_negotiation.cc:299] Negotiation successful
0806 03:50:53.440760 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260806 03:50:53.442659 5499 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48458:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 03:50:53.444442 5499 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 03:50:53.451166 6125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:50:53.447088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56062 (local address 127.11.253.194:38787)
0806 03:50:53.447299 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0806 03:50:53.447304 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 03:50:53.447315 (+ 11us) server_negotiation.cc:407] Connection header received
0806 03:50:53.447353 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:50:53.447355 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:50:53.447370 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:50:53.447448 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 03:50:53.447724 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:50:53.448659 (+ 935us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:50:53.450048 (+ 1389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:50:53.450751 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:50:53.450841 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:50:53.450904 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0806 03:50:53.451042 (+ 138us) server_negotiation.cc:299] Negotiation successful
0806 03:50:53.451081 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":179,"thread_start_us":92,"threads_started":1}
I20260806 03:50:53.451156 6122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:50:53.446984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.194:38787 (local address 127.0.0.1:56062)
0806 03:50:53.447137 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0806 03:50:53.447181 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0806 03:50:53.447247 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:50:53.447537 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:50:53.447540 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:50:53.447542 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 03:50:53.447618 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:50:53.447619 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:50:53.448762 (+ 1143us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:50:53.448763 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:50:53.449956 (+ 1193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:50:53.449958 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:50:53.450838 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:50:53.450839 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:50:53.450930 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:50:53.450945 (+ 15us) client_negotiation.cc:769] Sending connection context
0806 03:50:53.451015 (+ 70us) client_negotiation.cc:240] Negotiation successful
0806 03:50:53.451073 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"mutex_wait_us":7,"thread_start_us":47,"threads_started":1}
I20260806 03:50:53.451999 6123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:50:53.446990 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.193:45311 (local address 127.0.0.1:58424)
0806 03:50:53.447328 (+ 338us) negotiation.cc:107] Waiting for socket to connect
0806 03:50:53.447342 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 03:50:53.447400 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:50:53.447781 (+ 381us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:50:53.447782 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:50:53.447784 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 03:50:53.447844 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:50:53.447846 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:50:53.449336 (+ 1490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:50:53.449337 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:50:53.450388 (+ 1051us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:50:53.450390 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:50:53.451743 (+ 1353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:50:53.451744 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:50:53.451807 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:50:53.451811 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 03:50:53.451880 (+ 69us) client_negotiation.cc:240] Negotiation successful
0806 03:50:53.451921 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":312,"thread_start_us":110,"threads_started":1}
I20260806 03:50:53.452112 6127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:50:53.447220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.195:44125 (local address 127.0.0.1:52250)
0806 03:50:53.447426 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0806 03:50:53.447436 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 03:50:53.447477 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:50:53.448145 (+ 668us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:50:53.448146 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:50:53.448148 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 03:50:53.448186 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:50:53.448187 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:50:53.449535 (+ 1348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:50:53.449536 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:50:53.450780 (+ 1244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:50:53.450784 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:50:53.451851 (+ 1067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:50:53.451852 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:50:53.451923 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:50:53.451927 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 03:50:53.451982 (+ 55us) client_negotiation.cc:240] Negotiation successful
0806 03:50:53.452025 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":187,"mutex_wait_us":1,"thread_start_us":164,"threads_started":1}
I20260806 03:50:53.452035 5836 tablet_service.cc:1467] Processing CreateTablet for tablet f66e71098f8c4da2996b01168c0da424 (DEFAULT_TABLE table=test-table [id=c2f0d1699ff3408a86e8e65eb743aa33]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 03:50:53.452355 6126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:50:53.447121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58424 (local address 127.11.253.193:45311)
0806 03:50:53.447603 (+ 482us) server_negotiation.cc:206] Beginning negotiation
0806 03:50:53.447609 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 03:50:53.447620 (+ 11us) server_negotiation.cc:407] Connection header received
0806 03:50:53.447665 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:50:53.447667 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:50:53.447684 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:50:53.447775 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 03:50:53.447947 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:50:53.449218 (+ 1271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:50:53.450478 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:50:53.451658 (+ 1180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:50:53.451747 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:50:53.451810 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0806 03:50:53.451897 (+ 87us) server_negotiation.cc:299] Negotiation successful
0806 03:50:53.451946 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":439,"thread_start_us":96,"threads_started":1}
I20260806 03:50:53.452378 6128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:50:53.447524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52250 (local address 127.11.253.195:44125)
0806 03:50:53.447962 (+ 438us) server_negotiation.cc:206] Beginning negotiation
0806 03:50:53.447966 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 03:50:53.447976 (+ 10us) server_negotiation.cc:407] Connection header received
0806 03:50:53.448019 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:50:53.448022 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:50:53.448039 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:50:53.448134 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 03:50:53.448260 (+ 126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:50:53.449427 (+ 1167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:50:53.450886 (+ 1459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:50:53.451680 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:50:53.451760 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:50:53.451810 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0806 03:50:53.452241 (+ 431us) server_negotiation.cc:299] Negotiation successful
0806 03:50:53.452290 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":393,"thread_start_us":122,"threads_started":1}
I20260806 03:50:53.452436 5836 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f66e71098f8c4da2996b01168c0da424. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:50:53.453143 5667 tablet_service.cc:1467] Processing CreateTablet for tablet f66e71098f8c4da2996b01168c0da424 (DEFAULT_TABLE table=test-table [id=c2f0d1699ff3408a86e8e65eb743aa33]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 03:50:53.453122 6016 tablet_service.cc:1467] Processing CreateTablet for tablet f66e71098f8c4da2996b01168c0da424 (DEFAULT_TABLE table=test-table [id=c2f0d1699ff3408a86e8e65eb743aa33]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 03:50:53.453487 5667 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f66e71098f8c4da2996b01168c0da424. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:50:53.453487 6016 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f66e71098f8c4da2996b01168c0da424. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:50:53.455788 6129 tablet_bootstrap.cc:492] T f66e71098f8c4da2996b01168c0da424 P b632a244bf9c4689ad518228e3bf96d4: Bootstrap starting.
I20260806 03:50:53.456354 6129 tablet_bootstrap.cc:654] T f66e71098f8c4da2996b01168c0da424 P b632a244bf9c4689ad518228e3bf96d4: Neither blocks nor log segments found. Creating new log.
I20260806 03:50:53.456842 6129 log.cc:824] T f66e71098f8c4da2996b01168c0da424 P b632a244bf9c4689ad518228e3bf96d4: Log is configured to *not* fsync() on all Append() calls
I20260806 03:50:53.457324 6131 tablet_bootstrap.cc:492] T f66e71098f8c4da2996b01168c0da424 P 06093a7b77f5416c903d79d15353a38c: Bootstrap starting.
I20260806 03:50:53.457700 6129 tablet_bootstrap.cc:492] T f66e71098f8c4da2996b01168c0da424 P b632a244bf9c4689ad518228e3bf96d4: No bootstrap required, opened a new log
I20260806 03:50:53.457798 6129 ts_tablet_manager.cc:1397] T f66e71098f8c4da2996b01168c0da424 P b632a244bf9c4689ad518228e3bf96d4: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 03:50:53.457962 6132 tablet_bootstrap.cc:492] T f66e71098f8c4da2996b01168c0da424 P 89a6c7b7d5824849ba9579fdf6ab9b8c: Bootstrap starting.
I20260806 03:50:53.457994 6131 tablet_bootstrap.cc:654] T f66e71098f8c4da2996b01168c0da424 P 06093a7b77f5416c903d79d15353a38c: Neither blocks nor log segments found. Creating new log.
I20260806 03:50:53.458473 6131 log.cc:824] T f66e71098f8c4da2996b01168c0da424 P 06093a7b77f5416c903d79d15353a38c: Log is configured to *not* fsync() on all Append() calls
I20260806 03:50:53.458653 6132 tablet_bootstrap.cc:654] T f66e71098f8c4da2996b01168c0da424 P 89a6c7b7d5824849ba9579fdf6ab9b8c: Neither blocks nor log segments found. Creating new log.
I20260806 03:50:53.459259 6132 log.cc:824] T f66e71098f8c4da2996b01168c0da424 P 89a6c7b7d5824849ba9579fdf6ab9b8c: Log is configured to *not* fsync() on all Append() calls
I20260806 03:50:53.459296 6131 tablet_bootstrap.cc:492] T f66e71098f8c4da2996b01168c0da424 P 06093a7b77f5416c903d79d15353a38c: No bootstrap required, opened a new log
I20260806 03:50:53.459378 6131 ts_tablet_manager.cc:1397] T f66e71098f8c4da2996b01168c0da424 P 06093a7b77f5416c903d79d15353a38c: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260806 03:50:53.460104 6132 tablet_bootstrap.cc:492] T f66e71098f8c4da2996b01168c0da424 P 89a6c7b7d5824849ba9579fdf6ab9b8c: No bootstrap required, opened a new log
I20260806 03:50:53.460186 6132 ts_tablet_manager.cc:1397] T f66e71098f8c4da2996b01168c0da424 P 89a6c7b7d5824849ba9579fdf6ab9b8c: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 03:50:53.460269 6129 raft_consensus.cc:348] T f66e71098f8c4da2996b01168c0da424 P b632a244bf9c4689ad518228e3bf96d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89a6c7b7d5824849ba9579fdf6ab9b8c" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 45311 } } peers { permanent_uuid: "b632a244bf9c4689ad518228e3bf96d4" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 38787 } } peers { permanent_uuid: "06093a7b77f5416c903d79d15353a38c" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 44125 } }
I20260806 03:50:53.460492 6129 raft_consensus.cc:374] T f66e71098f8c4da2996b01168c0da424 P b632a244bf9c4689ad518228e3bf96d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:50:53.460542 6129 raft_consensus.cc:729] T f66e71098f8c4da2996b01168c0da424 P b632a244bf9c4689ad518228e3bf96d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b632a244bf9c4689ad518228e3bf96d4, State: Initialized, Role: FOLLOWER
I20260806 03:50:53.460695 6129 consensus_queue.cc:260] T f66e71098f8c4da2996b01168c0da424 P b632a244bf9c4689ad518228e3bf96d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89a6c7b7d5824849ba9579fdf6ab9b8c" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 45311 } } peers { permanent_uuid: "b632a244bf9c4689ad518228e3bf96d4" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 38787 } } peers { permanent_uuid: "06093a7b77f5416c903d79d15353a38c" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 44125 } }
I20260806 03:50:53.461010 6129 ts_tablet_manager.cc:1428] T f66e71098f8c4da2996b01168c0da424 P b632a244bf9c4689ad518228e3bf96d4: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260806 03:50:53.461145 5920 heartbeater.cc:503] Master 127.11.253.254:37975 was elected leader, sending a full tablet report...
I20260806 03:50:53.461171 6131 raft_consensus.cc:348] T f66e71098f8c4da2996b01168c0da424 P 06093a7b77f5416c903d79d15353a38c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89a6c7b7d5824849ba9579fdf6ab9b8c" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 45311 } } peers { permanent_uuid: "b632a244bf9c4689ad518228e3bf96d4" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 38787 } } peers { permanent_uuid: "06093a7b77f5416c903d79d15353a38c" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 44125 } }
I20260806 03:50:53.461373 6131 raft_consensus.cc:374] T f66e71098f8c4da2996b01168c0da424 P 06093a7b77f5416c903d79d15353a38c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:50:53.461423 6131 raft_consensus.cc:729] T f66e71098f8c4da2996b01168c0da424 P 06093a7b77f5416c903d79d15353a38c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06093a7b77f5416c903d79d15353a38c, State: Initialized, Role: FOLLOWER
I20260806 03:50:53.461563 6131 consensus_queue.cc:260] T f66e71098f8c4da2996b01168c0da424 P 06093a7b77f5416c903d79d15353a38c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89a6c7b7d5824849ba9579fdf6ab9b8c" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 45311 } } peers { permanent_uuid: "b632a244bf9c4689ad518228e3bf96d4" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 38787 } } peers { permanent_uuid: "06093a7b77f5416c903d79d15353a38c" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 44125 } }
I20260806 03:50:53.461887 6131 ts_tablet_manager.cc:1428] T f66e71098f8c4da2996b01168c0da424 P 06093a7b77f5416c903d79d15353a38c: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260806 03:50:53.461926 6108 heartbeater.cc:503] Master 127.11.253.254:37975 was elected leader, sending a full tablet report...
I20260806 03:50:53.462706 6132 raft_consensus.cc:348] T f66e71098f8c4da2996b01168c0da424 P 89a6c7b7d5824849ba9579fdf6ab9b8c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89a6c7b7d5824849ba9579fdf6ab9b8c" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 45311 } } peers { permanent_uuid: "b632a244bf9c4689ad518228e3bf96d4" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 38787 } } peers { permanent_uuid: "06093a7b77f5416c903d79d15353a38c" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 44125 } }
I20260806 03:50:53.462916 6132 raft_consensus.cc:374] T f66e71098f8c4da2996b01168c0da424 P 89a6c7b7d5824849ba9579fdf6ab9b8c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:50:53.462977 6132 raft_consensus.cc:729] T f66e71098f8c4da2996b01168c0da424 P 89a6c7b7d5824849ba9579fdf6ab9b8c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 89a6c7b7d5824849ba9579fdf6ab9b8c, State: Initialized, Role: FOLLOWER
I20260806 03:50:53.463145 6132 consensus_queue.cc:260] T f66e71098f8c4da2996b01168c0da424 P 89a6c7b7d5824849ba9579fdf6ab9b8c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89a6c7b7d5824849ba9579fdf6ab9b8c" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 45311 } } peers { permanent_uuid: "b632a244bf9c4689ad518228e3bf96d4" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 38787 } } peers { permanent_uuid: "06093a7b77f5416c903d79d15353a38c" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 44125 } }
I20260806 03:50:53.463465 6132 ts_tablet_manager.cc:1428] T f66e71098f8c4da2996b01168c0da424 P 89a6c7b7d5824849ba9579fdf6ab9b8c: Time spent starting tablet: real 0.003s user 0.000s sys 0.004s
I20260806 03:50:53.463493 5747 heartbeater.cc:503] Master 127.11.253.254:37975 was elected leader, sending a full tablet report...
W20260806 03:50:53.486044 5924 tablet.cc:2369] T f66e71098f8c4da2996b01168c0da424 P b632a244bf9c4689ad518228e3bf96d4: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 03:50:53.568337 5748 tablet.cc:2369] T f66e71098f8c4da2996b01168c0da424 P 89a6c7b7d5824849ba9579fdf6ab9b8c: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 03:50:53.664192 6109 tablet.cc:2369] T f66e71098f8c4da2996b01168c0da424 P 06093a7b77f5416c903d79d15353a38c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 03:50:53.696673 6138 raft_consensus.cc:482] T f66e71098f8c4da2996b01168c0da424 P 06093a7b77f5416c903d79d15353a38c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 03:50:53.696846 6138 raft_consensus.cc:504] T f66e71098f8c4da2996b01168c0da424 P 06093a7b77f5416c903d79d15353a38c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89a6c7b7d5824849ba9579fdf6ab9b8c" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 45311 } } peers { permanent_uuid: "b632a244bf9c4689ad518228e3bf96d4" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 38787 } } peers { permanent_uuid: "06093a7b77f5416c903d79d15353a38c" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 44125 } }
I20260806 03:50:53.697350 6138 leader_election.cc:288] T f66e71098f8c4da2996b01168c0da424 P 06093a7b77f5416c903d79d15353a38c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 89a6c7b7d5824849ba9579fdf6ab9b8c (127.11.253.193:45311), b632a244bf9c4689ad518228e3bf96d4 (127.11.253.194:38787)
I20260806 03:50:53.700690 6125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:50:53.697695 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.253.195:32817 (local address 127.11.253.194:38787)
0806 03:50:53.697755 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0806 03:50:53.697758 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 03:50:53.697843 (+ 85us) server_negotiation.cc:407] Connection header received
0806 03:50:53.697907 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:50:53.697909 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:50:53.697925 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:50:53.697987 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 03:50:53.698203 (+ 216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:50:53.698808 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:50:53.699844 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:50:53.700318 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:50:53.700397 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:50:53.700406 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0806 03:50:53.700569 (+ 163us) server_negotiation.cc:299] Negotiation successful
0806 03:50:53.700609 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260806 03:50:53.700702 6126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:50:53.697683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.253.195:55677 (local address 127.11.253.193:45311)
0806 03:50:53.697762 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0806 03:50:53.697764 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:50:53.697991 (+ 227us) server_negotiation.cc:407] Connection header received
0806 03:50:53.698025 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:50:53.698026 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:50:53.698041 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:50:53.698116 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 03:50:53.698364 (+ 248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:50:53.698823 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:50:53.699891 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:50:53.700241 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:50:53.700307 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:50:53.700315 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0806 03:50:53.700589 (+ 274us) server_negotiation.cc:299] Negotiation successful
0806 03:50:53.700632 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260806 03:50:53.700706 6111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:50:53.697597 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.194:38787 (local address 127.11.253.195:32817)
0806 03:50:53.697678 (+ 81us) negotiation.cc:107] Waiting for socket to connect
0806 03:50:53.697697 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 03:50:53.697753 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:50:53.698018 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:50:53.698020 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:50:53.698022 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 03:50:53.698110 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:50:53.698112 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:50:53.698911 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:50:53.698912 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:50:53.699732 (+ 820us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:50:53.699735 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:50:53.700406 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:50:53.700407 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:50:53.700484 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:50:53.700487 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 03:50:53.700530 (+ 43us) client_negotiation.cc:240] Negotiation successful
0806 03:50:53.700592 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":54,"mutex_wait_us":32}
I20260806 03:50:53.700730 6140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:50:53.697597 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.193:45311 (local address 127.11.253.195:55677)
0806 03:50:53.697845 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0806 03:50:53.697859 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 03:50:53.697957 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:50:53.698133 (+ 176us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:50:53.698133 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:50:53.698135 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 03:50:53.698234 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:50:53.698236 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:50:53.698940 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:50:53.698941 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:50:53.699767 (+ 826us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:50:53.699769 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:50:53.700350 (+ 581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:50:53.700351 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:50:53.700468 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:50:53.700482 (+ 14us) client_negotiation.cc:769] Sending connection context
0806 03:50:53.700569 (+ 87us) client_negotiation.cc:240] Negotiation successful
0806 03:50:53.700609 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":227,"mutex_wait_us":26,"thread_start_us":147,"threads_started":1}
I20260806 03:50:53.701136 5858 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f66e71098f8c4da2996b01168c0da424" candidate_uuid: "06093a7b77f5416c903d79d15353a38c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b632a244bf9c4689ad518228e3bf96d4" is_pre_election: true
I20260806 03:50:53.701351 5858 raft_consensus.cc:2454] T f66e71098f8c4da2996b01168c0da424 P b632a244bf9c4689ad518228e3bf96d4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06093a7b77f5416c903d79d15353a38c in term 0.
I20260806 03:50:53.701398 5690 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f66e71098f8c4da2996b01168c0da424" candidate_uuid: "06093a7b77f5416c903d79d15353a38c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "89a6c7b7d5824849ba9579fdf6ab9b8c" is_pre_election: true
I20260806 03:50:53.701576 5690 raft_consensus.cc:2454] T f66e71098f8c4da2996b01168c0da424 P 89a6c7b7d5824849ba9579fdf6ab9b8c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06093a7b77f5416c903d79d15353a38c in term 0.
I20260806 03:50:53.701594 5958 leader_election.cc:302] T f66e71098f8c4da2996b01168c0da424 P 06093a7b77f5416c903d79d15353a38c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06093a7b77f5416c903d79d15353a38c, b632a244bf9c4689ad518228e3bf96d4; no voters:
I20260806 03:50:53.701752 6138 raft_consensus.cc:2781] T f66e71098f8c4da2996b01168c0da424 P 06093a7b77f5416c903d79d15353a38c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 03:50:53.701866 6138 raft_consensus.cc:482] T f66e71098f8c4da2996b01168c0da424 P 06093a7b77f5416c903d79d15353a38c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 03:50:53.701938 6138 raft_consensus.cc:3037] T f66e71098f8c4da2996b01168c0da424 P 06093a7b77f5416c903d79d15353a38c [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:50:53.702754 6138 raft_consensus.cc:504] T f66e71098f8c4da2996b01168c0da424 P 06093a7b77f5416c903d79d15353a38c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89a6c7b7d5824849ba9579fdf6ab9b8c" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 45311 } } peers { permanent_uuid: "b632a244bf9c4689ad518228e3bf96d4" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 38787 } } peers { permanent_uuid: "06093a7b77f5416c903d79d15353a38c" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 44125 } }
I20260806 03:50:53.702961 6138 leader_election.cc:288] T f66e71098f8c4da2996b01168c0da424 P 06093a7b77f5416c903d79d15353a38c [CANDIDATE]: Term 1 election: Requested vote from peers 89a6c7b7d5824849ba9579fdf6ab9b8c (127.11.253.193:45311), b632a244bf9c4689ad518228e3bf96d4 (127.11.253.194:38787)
I20260806 03:50:53.703099 5690 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f66e71098f8c4da2996b01168c0da424" candidate_uuid: "06093a7b77f5416c903d79d15353a38c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "89a6c7b7d5824849ba9579fdf6ab9b8c"
I20260806 03:50:53.703107 5858 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f66e71098f8c4da2996b01168c0da424" candidate_uuid: "06093a7b77f5416c903d79d15353a38c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b632a244bf9c4689ad518228e3bf96d4"
I20260806 03:50:53.703184 5858 raft_consensus.cc:3037] T f66e71098f8c4da2996b01168c0da424 P b632a244bf9c4689ad518228e3bf96d4 [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:50:53.703186 5690 raft_consensus.cc:3037] T f66e71098f8c4da2996b01168c0da424 P 89a6c7b7d5824849ba9579fdf6ab9b8c [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:50:53.704152 5690 raft_consensus.cc:2454] T f66e71098f8c4da2996b01168c0da424 P 89a6c7b7d5824849ba9579fdf6ab9b8c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06093a7b77f5416c903d79d15353a38c in term 1.
I20260806 03:50:53.704152 5858 raft_consensus.cc:2454] T f66e71098f8c4da2996b01168c0da424 P b632a244bf9c4689ad518228e3bf96d4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06093a7b77f5416c903d79d15353a38c in term 1.
I20260806 03:50:53.704382 5958 leader_election.cc:302] T f66e71098f8c4da2996b01168c0da424 P 06093a7b77f5416c903d79d15353a38c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06093a7b77f5416c903d79d15353a38c, b632a244bf9c4689ad518228e3bf96d4; no voters:
I20260806 03:50:53.704502 6138 raft_consensus.cc:2781] T f66e71098f8c4da2996b01168c0da424 P 06093a7b77f5416c903d79d15353a38c [term 1 FOLLOWER]: Leader election won for term 1
I20260806 03:50:53.704725 6138 raft_consensus.cc:686] T f66e71098f8c4da2996b01168c0da424 P 06093a7b77f5416c903d79d15353a38c [term 1 LEADER]: Becoming Leader. State: Replica: 06093a7b77f5416c903d79d15353a38c, State: Running, Role: LEADER
I20260806 03:50:53.704845 6138 consensus_queue.cc:237] T f66e71098f8c4da2996b01168c0da424 P 06093a7b77f5416c903d79d15353a38c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89a6c7b7d5824849ba9579fdf6ab9b8c" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 45311 } } peers { permanent_uuid: "b632a244bf9c4689ad518228e3bf96d4" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 38787 } } peers { permanent_uuid: "06093a7b77f5416c903d79d15353a38c" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 44125 } }
I20260806 03:50:53.705794 5496 catalog_manager.cc:5495] T f66e71098f8c4da2996b01168c0da424 P 06093a7b77f5416c903d79d15353a38c reported cstate change: term changed from 0 to 1, leader changed from <none> to 06093a7b77f5416c903d79d15353a38c (127.11.253.195). New cstate: current_term: 1 leader_uuid: "06093a7b77f5416c903d79d15353a38c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89a6c7b7d5824849ba9579fdf6ab9b8c" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 45311 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b632a244bf9c4689ad518228e3bf96d4" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 38787 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "06093a7b77f5416c903d79d15353a38c" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 44125 } health_report { overall_health: HEALTHY } } }
I20260806 03:50:53.735001 6128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:50:53.732336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52260 (local address 127.11.253.195:44125)
0806 03:50:53.732407 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0806 03:50:53.732409 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:50:53.732417 (+ 8us) server_negotiation.cc:407] Connection header received
0806 03:50:53.732456 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:50:53.732456 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:50:53.732475 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:50:53.732546 (+ 71us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 03:50:53.732669 (+ 123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:50:53.733202 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:50:53.734184 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:50:53.734436 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:50:53.734515 (+ 79us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:50:53.734661 (+ 146us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 03:50:53.734763 (+ 102us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 03:50:53.734847 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0806 03:50:53.734891 (+ 44us) server_negotiation.cc:299] Negotiation successful
0806 03:50:53.734902 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260806 03:50:53.736999 5858 raft_consensus.cc:1261] T f66e71098f8c4da2996b01168c0da424 P b632a244bf9c4689ad518228e3bf96d4 [term 1 FOLLOWER]: Refusing update from remote peer 06093a7b77f5416c903d79d15353a38c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 03:50:53.737051 5690 raft_consensus.cc:1261] T f66e71098f8c4da2996b01168c0da424 P 89a6c7b7d5824849ba9579fdf6ab9b8c [term 1 FOLLOWER]: Refusing update from remote peer 06093a7b77f5416c903d79d15353a38c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 03:50:53.737442 6142 consensus_queue.cc:1035] T f66e71098f8c4da2996b01168c0da424 P 06093a7b77f5416c903d79d15353a38c [LEADER]: Connected to new peer: Peer: permanent_uuid: "b632a244bf9c4689ad518228e3bf96d4" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 38787 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 03:50:53.737633 6138 consensus_queue.cc:1035] T f66e71098f8c4da2996b01168c0da424 P 06093a7b77f5416c903d79d15353a38c [LEADER]: Connected to new peer: Peer: permanent_uuid: "89a6c7b7d5824849ba9579fdf6ab9b8c" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 45311 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 03:50:53.740257 6148 mvcc.cc:205] Tried to move back new op lower bound from 7315407887303610368 to 7315407887175700480. Current Snapshot: MvccSnapshot[applied={T|T < 7315407887303610368}]
I20260806 03:50:53.742627 5496 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48458:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 03:50:53.742738 5496 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48458:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 03:50:53.743506 5496 catalog_manager.cc:5782] T 00000000000000000000000000000000 P de8cab405fea49fbad8597dfce1c9827: Sending DeleteTablet for 3 replicas of tablet f66e71098f8c4da2996b01168c0da424
I20260806 03:50:53.743837 5836 tablet_service.cc:1514] Processing DeleteTablet for tablet f66e71098f8c4da2996b01168c0da424 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 03:50:53 UTC) from {username='kudu', principal='kudu/127.11.253.254@KRBTEST.COM'} at 127.0.0.1:56062
I20260806 03:50:53.743927 5667 tablet_service.cc:1514] Processing DeleteTablet for tablet f66e71098f8c4da2996b01168c0da424 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 03:50:53 UTC) from {username='kudu', principal='kudu/127.11.253.254@KRBTEST.COM'} at 127.0.0.1:58424
I20260806 03:50:53.743930 6016 tablet_service.cc:1514] Processing DeleteTablet for tablet f66e71098f8c4da2996b01168c0da424 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 03:50:53 UTC) from {username='kudu', principal='kudu/127.11.253.254@KRBTEST.COM'} at 127.0.0.1:52250
I20260806 03:50:53.744242 6159 tablet_replica.cc:330] T f66e71098f8c4da2996b01168c0da424 P b632a244bf9c4689ad518228e3bf96d4: stopping tablet replica
I20260806 03:50:53.744305 6160 tablet_replica.cc:330] T f66e71098f8c4da2996b01168c0da424 P 89a6c7b7d5824849ba9579fdf6ab9b8c: stopping tablet replica
I20260806 03:50:53.744364 12279 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskToVWRe/build/release/bin/kudu with pid 5579
I20260806 03:50:53.744406 6160 raft_consensus.cc:2229] T f66e71098f8c4da2996b01168c0da424 P 89a6c7b7d5824849ba9579fdf6ab9b8c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 03:50:53.744482 6159 raft_consensus.cc:2229] T f66e71098f8c4da2996b01168c0da424 P b632a244bf9c4689ad518228e3bf96d4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 03:50:53.744520 6160 raft_consensus.cc:2258] T f66e71098f8c4da2996b01168c0da424 P 89a6c7b7d5824849ba9579fdf6ab9b8c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 03:50:53.744699 6161 tablet_replica.cc:330] T f66e71098f8c4da2996b01168c0da424 P 06093a7b77f5416c903d79d15353a38c: stopping tablet replica
I20260806 03:50:53.744715 6159 raft_consensus.cc:2258] T f66e71098f8c4da2996b01168c0da424 P b632a244bf9c4689ad518228e3bf96d4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 03:50:53.744798 6161 raft_consensus.cc:2229] T f66e71098f8c4da2996b01168c0da424 P 06093a7b77f5416c903d79d15353a38c [term 1 LEADER]: Raft consensus shutting down.
I20260806 03:50:53.744895 6161 raft_consensus.cc:2258] T f66e71098f8c4da2996b01168c0da424 P 06093a7b77f5416c903d79d15353a38c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 03:50:53.747090 6159 ts_tablet_manager.cc:1905] T f66e71098f8c4da2996b01168c0da424 P b632a244bf9c4689ad518228e3bf96d4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 03:50:53.748176 6161 ts_tablet_manager.cc:1905] T f66e71098f8c4da2996b01168c0da424 P 06093a7b77f5416c903d79d15353a38c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 03:50:53.748953 6159 ts_tablet_manager.cc:1918] T f66e71098f8c4da2996b01168c0da424 P b632a244bf9c4689ad518228e3bf96d4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 03:50:53.749104 6159 log.cc:1196] T f66e71098f8c4da2996b01168c0da424 P b632a244bf9c4689ad518228e3bf96d4: Deleting WAL directory at /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785988004637995-12279-0/minicluster-data/ts-1/wal/wals/f66e71098f8c4da2996b01168c0da424
I20260806 03:50:53.749459 6159 ts_tablet_manager.cc:1939] T f66e71098f8c4da2996b01168c0da424 P b632a244bf9c4689ad518228e3bf96d4: Deleting consensus metadata
I20260806 03:50:53.749970 6161 ts_tablet_manager.cc:1918] T f66e71098f8c4da2996b01168c0da424 P 06093a7b77f5416c903d79d15353a38c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 03:50:53.750005 5481 catalog_manager.cc:4841] TS b632a244bf9c4689ad518228e3bf96d4 (127.11.253.194:38787): tablet f66e71098f8c4da2996b01168c0da424 (table test-table [id=c2f0d1699ff3408a86e8e65eb743aa33]) successfully deleted
I20260806 03:50:53.750043 6161 log.cc:1196] T f66e71098f8c4da2996b01168c0da424 P 06093a7b77f5416c903d79d15353a38c: Deleting WAL directory at /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785988004637995-12279-0/minicluster-data/ts-2/wal/wals/f66e71098f8c4da2996b01168c0da424
I20260806 03:50:53.750336 6161 ts_tablet_manager.cc:1939] T f66e71098f8c4da2996b01168c0da424 P 06093a7b77f5416c903d79d15353a38c: Deleting consensus metadata
I20260806 03:50:53.750873 5480 catalog_manager.cc:4841] TS 06093a7b77f5416c903d79d15353a38c (127.11.253.195:44125): tablet f66e71098f8c4da2996b01168c0da424 (table test-table [id=c2f0d1699ff3408a86e8e65eb743aa33]) successfully deleted
W20260806 03:50:53.780332 5480 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.253.193:45311 (error 108)
I20260806 03:50:53.781010 6127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:50:53.780855 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.193:45311 (local address 127.0.0.1:58432)
0806 03:50:53.780922 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0806 03:50:53.780968 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.253.193:45311: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260806 03:50:53.781105 12279 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskToVWRe/build/release/bin/kudu with pid 5754
W20260806 03:50:53.788481 5480 catalog_manager.cc:4584] TS 89a6c7b7d5824849ba9579fdf6ab9b8c (127.11.253.193:45311): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet f66e71098f8c4da2996b01168c0da424: Network error: Client connection negotiation failed: client connection to 127.11.253.193:45311: connect: Connection refused (error 111)
I20260806 03:50:53.794132 12279 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskToVWRe/build/release/bin/kudu with pid 5933
I20260806 03:50:53.806324 12279 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskToVWRe/build/release/bin/kudu with pid 5455
2026-08-06T03:50:53Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2435 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-06T03:50:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T03:50:53Z Disabled control of system clock
I20260806 03:50:53.839295 12279 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskToVWRe/build/release/bin/kudu
/tmp/dist-test-taskToVWRe/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785988004637995-12279-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785988004637995-12279-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785988004637995-12279-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785988004637995-12279-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.253.254:43933
--webserver_interface=127.11.253.254
--webserver_port=0
--builtin_ntp_servers=127.11.253.212:34079
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.253.254:43933
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 03:50:53.923053 6168 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:50:53.923281 6168 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:50:53.923303 6168 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:50:53.924481 6168 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 03:50:53.924526 6168 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:50:53.924539 6168 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:50:53.924551 6168 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 03:50:53.924564 6168 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 03:50:53.925839 6168 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.253.212:34079
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785988004637995-12279-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785988004637995-12279-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.253.254:43933
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.253.254:43933
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785988004637995-12279-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.253.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6168
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785988004637995-12279-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:50:53.926079 6168 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:50:53.926323 6168 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:50:53.931427 6177 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:50:53.931437 6174 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:50:53.931471 6181 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:50:53.931945 6168 server_base.cc:1034] running on GCE node
I20260806 03:50:53.932173 6168 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:50:53.932418 6168 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:50:53.933568 6168 hybrid_clock.cc:648] HybridClock initialized: now 1785988253933551 us; error 31 us; skew 500 ppm
I20260806 03:50:53.934918 6168 webserver.cc:460] Webserver started at http://127.11.253.254:42763/ using document root <none> and password file <none>
I20260806 03:50:53.935119 6168 fs_manager.cc:359] Metadata directory not provided
I20260806 03:50:53.935158 6168 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:50:53.935249 6168 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:50:53.936271 6168 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785988004637995-12279-0/minicluster-data/master-0/data/instance:
uuid: "fb96b8b0b07140128e93e2e701c08717"
format_stamp: "Formatted at 2026-08-06 03:50:53 on dist-test-slave-6ntk"
server_key: "0fda4992bd956aa51f27905ba2ca2c95"
server_key_iv: "e8cb77278e7fabd27c02223b48d077f6"
server_key_version: "encryptionkey@0"
I20260806 03:50:53.936591 6168 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785988004637995-12279-0/minicluster-data/master-0/wal/instance:
uuid: "fb96b8b0b07140128e93e2e701c08717"
format_stamp: "Formatted at 2026-08-06 03:50:53 on dist-test-slave-6ntk"
server_key: "0fda4992bd956aa51f27905ba2ca2c95"
server_key_iv: "e8cb77278e7fabd27c02223b48d077f6"
server_key_version: "encryptionkey@0"
I20260806 03:50:53.937835 6168 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 03:50:53.938637 6187 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:50:53.938793 6168 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 03:50:53.938886 6168 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785988004637995-12279-0/minicluster-data/master-0/data,/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785988004637995-12279-0/minicluster-data/master-0/wal
uuid: "fb96b8b0b07140128e93e2e701c08717"
format_stamp: "Formatted at 2026-08-06 03:50:53 on dist-test-slave-6ntk"
server_key: "0fda4992bd956aa51f27905ba2ca2c95"
server_key_iv: "e8cb77278e7fabd27c02223b48d077f6"
server_key_version: "encryptionkey@0"
I20260806 03:50:53.938942 6168 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785988004637995-12279-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785988004637995-12279-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785988004637995-12279-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:50:53.956278 6168 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:50:53.956581 6168 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:50:53.956701 6168 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:50:53.960722 6168 rpc_server.cc:307] RPC server started. Bound to: 127.11.253.254:43933
I20260806 03:50:53.960824 6269 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.253.254:43933 every 8 connection(s)
I20260806 03:50:53.961149 6168 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785988004637995-12279-0/minicluster-data/master-0/data/info.pb
I20260806 03:50:53.962234 6271 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:50:53.964200 12279 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskToVWRe/build/release/bin/kudu as pid 6168
I20260806 03:50:53.964320 12279 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785988004637995-12279-0/minicluster-data/master-0/wal/instance
I20260806 03:50:53.964589 12279 external_mini_cluster.cc:1428] Setting key 25f063b897bf408f350dba7188e006bf
I20260806 03:50:53.964816 6271 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fb96b8b0b07140128e93e2e701c08717: Bootstrap starting.
I20260806 03:50:53.965566 6271 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fb96b8b0b07140128e93e2e701c08717: Neither blocks nor log segments found. Creating new log.
I20260806 03:50:53.965973 6271 log.cc:824] T 00000000000000000000000000000000 P fb96b8b0b07140128e93e2e701c08717: Log is configured to *not* fsync() on all Append() calls
I20260806 03:50:53.966861 6271 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fb96b8b0b07140128e93e2e701c08717: No bootstrap required, opened a new log
I20260806 03:50:53.969213 6271 raft_consensus.cc:348] T 00000000000000000000000000000000 P fb96b8b0b07140128e93e2e701c08717 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb96b8b0b07140128e93e2e701c08717" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 43933 } }
I20260806 03:50:53.969398 6271 raft_consensus.cc:374] T 00000000000000000000000000000000 P fb96b8b0b07140128e93e2e701c08717 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:50:53.969434 6271 raft_consensus.cc:729] T 00000000000000000000000000000000 P fb96b8b0b07140128e93e2e701c08717 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb96b8b0b07140128e93e2e701c08717, State: Initialized, Role: FOLLOWER
I20260806 03:50:53.969581 6271 consensus_queue.cc:260] T 00000000000000000000000000000000 P fb96b8b0b07140128e93e2e701c08717 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb96b8b0b07140128e93e2e701c08717" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 43933 } }
I20260806 03:50:53.969671 6271 raft_consensus.cc:388] T 00000000000000000000000000000000 P fb96b8b0b07140128e93e2e701c08717 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 03:50:53.969712 6271 raft_consensus.cc:482] T 00000000000000000000000000000000 P fb96b8b0b07140128e93e2e701c08717 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 03:50:53.969760 6271 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fb96b8b0b07140128e93e2e701c08717 [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:50:53.970360 6271 raft_consensus.cc:504] T 00000000000000000000000000000000 P fb96b8b0b07140128e93e2e701c08717 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb96b8b0b07140128e93e2e701c08717" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 43933 } }
I20260806 03:50:53.970500 6271 leader_election.cc:302] T 00000000000000000000000000000000 P fb96b8b0b07140128e93e2e701c08717 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fb96b8b0b07140128e93e2e701c08717; no voters:
I20260806 03:50:53.970700 6271 leader_election.cc:288] T 00000000000000000000000000000000 P fb96b8b0b07140128e93e2e701c08717 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 03:50:53.970752 6279 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fb96b8b0b07140128e93e2e701c08717 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 03:50:53.970916 6279 raft_consensus.cc:686] T 00000000000000000000000000000000 P fb96b8b0b07140128e93e2e701c08717 [term 1 LEADER]: Becoming Leader. State: Replica: fb96b8b0b07140128e93e2e701c08717, State: Running, Role: LEADER
I20260806 03:50:53.971051 6271 sys_catalog.cc:564] T 00000000000000000000000000000000 P fb96b8b0b07140128e93e2e701c08717 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 03:50:53.971042 6279 consensus_queue.cc:237] T 00000000000000000000000000000000 P fb96b8b0b07140128e93e2e701c08717 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb96b8b0b07140128e93e2e701c08717" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 43933 } }
I20260806 03:50:53.971517 6282 sys_catalog.cc:455] T 00000000000000000000000000000000 P fb96b8b0b07140128e93e2e701c08717 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fb96b8b0b07140128e93e2e701c08717" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb96b8b0b07140128e93e2e701c08717" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 43933 } } }
I20260806 03:50:53.971619 6282 sys_catalog.cc:458] T 00000000000000000000000000000000 P fb96b8b0b07140128e93e2e701c08717 [sys.catalog]: This master's current role is: LEADER
I20260806 03:50:53.971504 6279 sys_catalog.cc:455] T 00000000000000000000000000000000 P fb96b8b0b07140128e93e2e701c08717 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fb96b8b0b07140128e93e2e701c08717. Latest consensus state: current_term: 1 leader_uuid: "fb96b8b0b07140128e93e2e701c08717" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb96b8b0b07140128e93e2e701c08717" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 43933 } } }
I20260806 03:50:53.971820 6277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:50:53.965472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33024 (local address 127.11.253.254:43933)
0806 03:50:53.965734 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0806 03:50:53.965745 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 03:50:53.965785 (+ 40us) server_negotiation.cc:407] Connection header received
0806 03:50:53.966387 (+ 602us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:50:53.966395 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:50:53.966457 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:50:53.966626 (+ 169us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:50:53.966841 (+ 215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:50:53.968258 (+ 1417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:50:53.969034 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:50:53.969635 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:50:53.969768 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:50:53.970766 (+ 998us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:50:53.970771 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:50:53.970771 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 03:50:53.971055 (+ 284us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:50:53.971105 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 03:50:53.971121 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:50:53.971399 (+ 278us) server_negotiation.cc:1035] Waiting for connection context
0806 03:50:53.971440 (+ 41us) server_negotiation.cc:299] Negotiation successful
0806 03:50:53.971543 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":161,"thread_start_us":99,"threads_started":1}
I20260806 03:50:53.971866 6279 sys_catalog.cc:458] T 00000000000000000000000000000000 P fb96b8b0b07140128e93e2e701c08717 [sys.catalog]: This master's current role is: LEADER
I20260806 03:50:53.972527 6287 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 03:50:53.973138 6287 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 03:50:53.974714 6287 catalog_manager.cc:1347] Generated new cluster ID: e3a5171e59d1405c95f86d7214662bb0
I20260806 03:50:53.974766 6287 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 03:50:53.983031 6287 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 03:50:53.983815 6287 catalog_manager.cc:1504] Loading token signing keys...
I20260806 03:50:53.989338 6287 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fb96b8b0b07140128e93e2e701c08717: Generated new TSK 0
I20260806 03:50:53.989539 6287 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 03:50:53.991206 12279 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskToVWRe/build/release/bin/kudu
/tmp/dist-test-taskToVWRe/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785988004637995-12279-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785988004637995-12279-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785988004637995-12279-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785988004637995-12279-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.253.193:0
--local_ip_for_outbound_sockets=127.11.253.193
--webserver_interface=127.11.253.193
--webserver_port=0
--tserver_master_addrs=127.11.253.254:43933
--builtin_ntp_servers=127.11.253.212:34079
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 03:50:54.078354 6303 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:50:54.078599 6303 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:50:54.078639 6303 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:50:54.079821 6303 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:50:54.079896 6303 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:50:54.079942 6303 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.253.193
I20260806 03:50:54.081362 6303 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.253.212:34079
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785988004637995-12279-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785988004637995-12279-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.253.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785988004637995-12279-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.253.193
--webserver_port=0
--tserver_master_addrs=127.11.253.254:43933
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6303
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.253.193
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785988004637995-12279-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:50:54.081640 6303 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:50:54.082023 6303 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:50:54.088193 6310 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:50:54.088351 6309 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:50:54.088338 6303 server_base.cc:1034] running on GCE node
W20260806 03:50:54.088172 6313 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:50:54.088649 6303 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:50:54.088907 6303 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:50:54.090086 6303 hybrid_clock.cc:648] HybridClock initialized: now 1785988254090060 us; error 44 us; skew 500 ppm
I20260806 03:50:54.091452 6303 webserver.cc:460] Webserver started at http://127.11.253.193:44799/ using document root <none> and password file <none>
I20260806 03:50:54.091733 6303 fs_manager.cc:359] Metadata directory not provided
I20260806 03:50:54.091809 6303 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:50:54.091974 6303 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:50:54.093127 6303 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785988004637995-12279-0/minicluster-data/ts-0/data/instance:
uuid: "5950ce84fd61420d810ee844d602b2e3"
format_stamp: "Formatted at 2026-08-06 03:50:54 on dist-test-slave-6ntk"
server_key: "ce66a2f3c019a5c32f84ac76cd4870ca"
server_key_iv: "df5385c6d2c61d7714069b2e7a7782e7"
server_key_version: "encryptionkey@0"
I20260806 03:50:54.093495 6303 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785988004637995-12279-0/minicluster-data/ts-0/wal/instance:
uuid: "5950ce84fd61420d810ee844d602b2e3"
format_stamp: "Formatted at 2026-08-06 03:50:54 on dist-test-slave-6ntk"
server_key: "ce66a2f3c019a5c32f84ac76cd4870ca"
server_key_iv: "df5385c6d2c61d7714069b2e7a7782e7"
server_key_version: "encryptionkey@0"
I20260806 03:50:54.094995 6303 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 03:50:54.095965 6320 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:50:54.096141 6303 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 03:50:54.096248 6303 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785988004637995-12279-0/minicluster-data/ts-0/data,/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785988004637995-12279-0/minicluster-data/ts-0/wal
uuid: "5950ce84fd61420d810ee844d602b2e3"
format_stamp: "Formatted at 2026-08-06 03:50:54 on dist-test-slave-6ntk"
server_key: "ce66a2f3c019a5c32f84ac76cd4870ca"
server_key_iv: "df5385c6d2c61d7714069b2e7a7782e7"
server_key_version: "encryptionkey@0"
I20260806 03:50:54.096334 6303 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785988004637995-12279-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785988004637995-12279-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785988004637995-12279-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:50:54.121060 6303 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:50:54.121448 6303 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:50:54.121619 6303 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:50:54.121946 6303 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 03:50:54.122325 6303 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 03:50:54.122387 6303 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 03:50:54.122437 6303 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 03:50:54.122464 6303 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 03:50:54.128535 6303 rpc_server.cc:307] RPC server started. Bound to: 127.11.253.193:38625
I20260806 03:50:54.128659 6465 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.253.193:38625 every 8 connection(s)
I20260806 03:50:54.128966 6303 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785988004637995-12279-0/minicluster-data/ts-0/data/info.pb
I20260806 03:50:54.134567 6277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:50:54.129794 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.253.193:38627 (local address 127.11.253.254:43933)
0806 03:50:54.129857 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0806 03:50:54.129859 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:50:54.130091 (+ 232us) server_negotiation.cc:407] Connection header received
0806 03:50:54.130703 (+ 612us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:50:54.130705 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:50:54.130717 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:50:54.130774 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:50:54.131522 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:50:54.132039 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:50:54.133069 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:50:54.133271 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:50:54.133351 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:50:54.134253 (+ 902us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:50:54.134254 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:50:54.134254 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 03:50:54.134259 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:50:54.134274 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 03:50:54.134277 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:50:54.134325 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0806 03:50:54.134458 (+ 133us) server_negotiation.cc:299] Negotiation successful
0806 03:50:54.134484 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260806 03:50:54.134804 6468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:50:54.129752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:43933 (local address 127.11.253.193:38627)
0806 03:50:54.129947 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0806 03:50:54.129992 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 03:50:54.130582 (+ 590us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:50:54.130847 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:50:54.130852 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:50:54.130863 (+ 11us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:50:54.131405 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:50:54.131410 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:50:54.132145 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:50:54.132146 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:50:54.132970 (+ 824us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:50:54.132972 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:50:54.133363 (+ 391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:50:54.133364 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:50:54.133509 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:50:54.134103 (+ 594us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 03:50:54.134104 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:50:54.134134 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 03:50:54.134158 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:50:54.134369 (+ 211us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:50:54.134375 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:50:54.134378 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 03:50:54.134458 (+ 80us) client_negotiation.cc:240] Negotiation successful
0806 03:50:54.134568 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":65,"threads_started":1}
I20260806 03:50:54.135385 6466 heartbeater.cc:347] Connected to a master server at 127.11.253.254:43933
I20260806 03:50:54.135489 6466 heartbeater.cc:464] Registering TS with master...
I20260806 03:50:54.135672 6466 heartbeater.cc:511] Master 127.11.253.254:43933 requested a full tablet report, sending...
I20260806 03:50:54.136175 6215 ts_manager.cc:194] Registered new tserver with Master: 5950ce84fd61420d810ee844d602b2e3 (127.11.253.193:38625)
I20260806 03:50:54.136765 6215 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.11.253.193:38627
I20260806 03:50:54.138063 12279 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskToVWRe/build/release/bin/kudu as pid 6303
I20260806 03:50:54.138187 12279 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785988004637995-12279-0/minicluster-data/ts-0/wal/instance
I20260806 03:50:54.138450 12279 external_mini_cluster.cc:1428] Setting key e44c88d9ea338fe905ae865ce7625ae0
I20260806 03:50:54.139524 12279 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskToVWRe/build/release/bin/kudu
/tmp/dist-test-taskToVWRe/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785988004637995-12279-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785988004637995-12279-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785988004637995-12279-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785988004637995-12279-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.253.194:0
--local_ip_for_outbound_sockets=127.11.253.194
--webserver_interface=127.11.253.194
--webserver_port=0
--tserver_master_addrs=127.11.253.254:43933
--builtin_ntp_servers=127.11.253.212:34079
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 03:50:54.227803 6469 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:50:54.228025 6469 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:50:54.228049 6469 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:50:54.229126 6469 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:50:54.229161 6469 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:50:54.229185 6469 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.253.194
I20260806 03:50:54.230588 6469 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.253.212:34079
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785988004637995-12279-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785988004637995-12279-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.253.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785988004637995-12279-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.253.194
--webserver_port=0
--tserver_master_addrs=127.11.253.254:43933
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6469
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.253.194
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785988004637995-12279-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:50:54.230782 6469 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:50:54.231042 6469 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:50:54.236182 6474 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:50:54.236171 6476 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:50:54.236151 6480 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:50:54.236325 6469 server_base.cc:1034] running on GCE node
I20260806 03:50:54.236683 6469 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:50:54.236941 6469 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:50:54.238092 6469 hybrid_clock.cc:648] HybridClock initialized: now 1785988254238060 us; error 43 us; skew 500 ppm
I20260806 03:50:54.239456 6469 webserver.cc:460] Webserver started at http://127.11.253.194:37579/ using document root <none> and password file <none>
I20260806 03:50:54.239705 6469 fs_manager.cc:359] Metadata directory not provided
I20260806 03:50:54.239753 6469 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:50:54.239953 6469 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:50:54.241091 6469 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785988004637995-12279-0/minicluster-data/ts-1/data/instance:
uuid: "1b860db760ba493fbc9ac415b613f536"
format_stamp: "Formatted at 2026-08-06 03:50:54 on dist-test-slave-6ntk"
server_key: "f54f0513ff0a5e2a2b74bf96f39d36c1"
server_key_iv: "dcd4e92fd6352a98d7f0a57aa2eb811d"
server_key_version: "encryptionkey@0"
I20260806 03:50:54.241479 6469 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785988004637995-12279-0/minicluster-data/ts-1/wal/instance:
uuid: "1b860db760ba493fbc9ac415b613f536"
format_stamp: "Formatted at 2026-08-06 03:50:54 on dist-test-slave-6ntk"
server_key: "f54f0513ff0a5e2a2b74bf96f39d36c1"
server_key_iv: "dcd4e92fd6352a98d7f0a57aa2eb811d"
server_key_version: "encryptionkey@0"
I20260806 03:50:54.242862 6469 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 03:50:54.243643 6490 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:50:54.243834 6469 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 03:50:54.243932 6469 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785988004637995-12279-0/minicluster-data/ts-1/data,/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785988004637995-12279-0/minicluster-data/ts-1/wal
uuid: "1b860db760ba493fbc9ac415b613f536"
format_stamp: "Formatted at 2026-08-06 03:50:54 on dist-test-slave-6ntk"
server_key: "f54f0513ff0a5e2a2b74bf96f39d36c1"
server_key_iv: "dcd4e92fd6352a98d7f0a57aa2eb811d"
server_key_version: "encryptionkey@0"
I20260806 03:50:54.244007 6469 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785988004637995-12279-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785988004637995-12279-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785988004637995-12279-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:50:54.257138 6469 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:50:54.257484 6469 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:50:54.257644 6469 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:50:54.257920 6469 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 03:50:54.258293 6469 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 03:50:54.258328 6469 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 03:50:54.258373 6469 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 03:50:54.258399 6469 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 03:50:54.264340 6469 rpc_server.cc:307] RPC server started. Bound to: 127.11.253.194:36055
I20260806 03:50:54.264418 6640 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.253.194:36055 every 8 connection(s)
I20260806 03:50:54.264767 6469 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785988004637995-12279-0/minicluster-data/ts-1/data/info.pb
I20260806 03:50:54.265709 12279 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskToVWRe/build/release/bin/kudu as pid 6469
I20260806 03:50:54.265818 12279 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785988004637995-12279-0/minicluster-data/ts-1/wal/instance
I20260806 03:50:54.266078 12279 external_mini_cluster.cc:1428] Setting key df652f39d5207400015e95bcd9b71ceb
I20260806 03:50:54.267068 12279 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskToVWRe/build/release/bin/kudu
/tmp/dist-test-taskToVWRe/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785988004637995-12279-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785988004637995-12279-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785988004637995-12279-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785988004637995-12279-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.253.195:0
--local_ip_for_outbound_sockets=127.11.253.195
--webserver_interface=127.11.253.195
--webserver_port=0
--tserver_master_addrs=127.11.253.254:43933
--builtin_ntp_servers=127.11.253.212:34079
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260806 03:50:54.270663 6277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:50:54.265587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.253.194:43593 (local address 127.11.253.254:43933)
0806 03:50:54.265665 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0806 03:50:54.265668 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 03:50:54.265937 (+ 269us) server_negotiation.cc:407] Connection header received
0806 03:50:54.266608 (+ 671us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:50:54.266609 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:50:54.266622 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:50:54.266679 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:50:54.267492 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:50:54.267999 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:50:54.269091 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:50:54.269316 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:50:54.269400 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:50:54.270395 (+ 995us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:50:54.270396 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:50:54.270397 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 03:50:54.270401 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:50:54.270418 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 03:50:54.270422 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:50:54.270476 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0806 03:50:54.270556 (+ 80us) server_negotiation.cc:299] Negotiation successful
0806 03:50:54.270583 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260806 03:50:54.270890 6644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:50:54.265582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:43933 (local address 127.11.253.194:43593)
0806 03:50:54.265782 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0806 03:50:54.265824 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 03:50:54.266500 (+ 676us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:50:54.266759 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:50:54.266763 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:50:54.266774 (+ 11us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:50:54.267397 (+ 623us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:50:54.267402 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:50:54.268100 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:50:54.268101 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:50:54.268990 (+ 889us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:50:54.268993 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:50:54.269408 (+ 415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:50:54.269409 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:50:54.269558 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:50:54.270225 (+ 667us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 03:50:54.270226 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:50:54.270256 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 03:50:54.270286 (+ 30us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:50:54.270479 (+ 193us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:50:54.270485 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:50:54.270488 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 03:50:54.270550 (+ 62us) client_negotiation.cc:240] Negotiation successful
0806 03:50:54.270653 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":105,"threads_started":1}
I20260806 03:50:54.271341 6641 heartbeater.cc:347] Connected to a master server at 127.11.253.254:43933
I20260806 03:50:54.271440 6641 heartbeater.cc:464] Registering TS with master...
I20260806 03:50:54.271625 6641 heartbeater.cc:511] Master 127.11.253.254:43933 requested a full tablet report, sending...
I20260806 03:50:54.272001 6215 ts_manager.cc:194] Registered new tserver with Master: 1b860db760ba493fbc9ac415b613f536 (127.11.253.194:36055)
I20260806 03:50:54.272424 6215 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.11.253.194:43593
W20260806 03:50:54.352451 6645 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:50:54.352721 6645 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:50:54.352743 6645 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:50:54.353834 6645 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:50:54.353884 6645 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:50:54.353910 6645 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.253.195
I20260806 03:50:54.355180 6645 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.253.212:34079
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785988004637995-12279-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785988004637995-12279-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.253.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785988004637995-12279-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.253.195
--webserver_port=0
--tserver_master_addrs=127.11.253.254:43933
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6645
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.253.195
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785988004637995-12279-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:50:54.355368 6645 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:50:54.355626 6645 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:50:54.361011 6652 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:50:54.361014 6651 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:50:54.361044 6645 server_base.cc:1034] running on GCE node
W20260806 03:50:54.361042 6654 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:50:54.361447 6645 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:50:54.361713 6645 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:50:54.362877 6645 hybrid_clock.cc:648] HybridClock initialized: now 1785988254362863 us; error 36 us; skew 500 ppm
I20260806 03:50:54.364135 6645 webserver.cc:460] Webserver started at http://127.11.253.195:44255/ using document root <none> and password file <none>
I20260806 03:50:54.364405 6645 fs_manager.cc:359] Metadata directory not provided
I20260806 03:50:54.364475 6645 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:50:54.364629 6645 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:50:54.365741 6645 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785988004637995-12279-0/minicluster-data/ts-2/data/instance:
uuid: "d936ab250f71410aa88c0e9f4e9aaa85"
format_stamp: "Formatted at 2026-08-06 03:50:54 on dist-test-slave-6ntk"
server_key: "9f9b00a7b8d7c2706d092463f088ed98"
server_key_iv: "897ebb5e5bf4c550f92ba2bb40ddb256"
server_key_version: "encryptionkey@0"
I20260806 03:50:54.366165 6645 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785988004637995-12279-0/minicluster-data/ts-2/wal/instance:
uuid: "d936ab250f71410aa88c0e9f4e9aaa85"
format_stamp: "Formatted at 2026-08-06 03:50:54 on dist-test-slave-6ntk"
server_key: "9f9b00a7b8d7c2706d092463f088ed98"
server_key_iv: "897ebb5e5bf4c550f92ba2bb40ddb256"
server_key_version: "encryptionkey@0"
I20260806 03:50:54.367488 6645 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 03:50:54.368230 6662 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:50:54.368456 6645 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 03:50:54.368566 6645 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785988004637995-12279-0/minicluster-data/ts-2/data,/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785988004637995-12279-0/minicluster-data/ts-2/wal
uuid: "d936ab250f71410aa88c0e9f4e9aaa85"
format_stamp: "Formatted at 2026-08-06 03:50:54 on dist-test-slave-6ntk"
server_key: "9f9b00a7b8d7c2706d092463f088ed98"
server_key_iv: "897ebb5e5bf4c550f92ba2bb40ddb256"
server_key_version: "encryptionkey@0"
I20260806 03:50:54.368640 6645 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785988004637995-12279-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785988004637995-12279-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785988004637995-12279-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:50:54.407167 6645 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:50:54.407541 6645 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:50:54.407707 6645 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:50:54.407969 6645 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 03:50:54.408340 6645 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 03:50:54.408399 6645 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 03:50:54.408442 6645 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 03:50:54.408468 6645 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 03:50:54.414561 6645 rpc_server.cc:307] RPC server started. Bound to: 127.11.253.195:34213
I20260806 03:50:54.414605 6824 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.253.195:34213 every 8 connection(s)
I20260806 03:50:54.414943 6645 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785988004637995-12279-0/minicluster-data/ts-2/data/info.pb
I20260806 03:50:54.420683 6277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:50:54.415744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.253.195:58563 (local address 127.11.253.254:43933)
0806 03:50:54.415863 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0806 03:50:54.415865 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:50:54.416086 (+ 221us) server_negotiation.cc:407] Connection header received
0806 03:50:54.416664 (+ 578us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:50:54.416665 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:50:54.416679 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:50:54.416729 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:50:54.417465 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:50:54.418068 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:50:54.419108 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:50:54.419328 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:50:54.419410 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:50:54.420381 (+ 971us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:50:54.420383 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:50:54.420383 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 03:50:54.420388 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:50:54.420405 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 03:50:54.420409 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:50:54.420454 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0806 03:50:54.420566 (+ 112us) server_negotiation.cc:299] Negotiation successful
0806 03:50:54.420595 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260806 03:50:54.420943 6830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:50:54.415732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:43933 (local address 127.11.253.195:58563)
0806 03:50:54.415961 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0806 03:50:54.415997 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0806 03:50:54.416547 (+ 550us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:50:54.416807 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:50:54.416812 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:50:54.416822 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:50:54.417367 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:50:54.417371 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:50:54.418171 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:50:54.418172 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:50:54.419003 (+ 831us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:50:54.419005 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:50:54.419417 (+ 412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:50:54.419419 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:50:54.419568 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:50:54.420148 (+ 580us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 03:50:54.420149 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:50:54.420179 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 03:50:54.420205 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:50:54.420468 (+ 263us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:50:54.420474 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:50:54.420477 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 03:50:54.420568 (+ 91us) client_negotiation.cc:240] Negotiation successful
0806 03:50:54.420678 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":91,"threads_started":1}
I20260806 03:50:54.421552 6826 heartbeater.cc:347] Connected to a master server at 127.11.253.254:43933
I20260806 03:50:54.421644 6826 heartbeater.cc:464] Registering TS with master...
I20260806 03:50:54.421810 6826 heartbeater.cc:511] Master 127.11.253.254:43933 requested a full tablet report, sending...
I20260806 03:50:54.422201 6215 ts_manager.cc:194] Registered new tserver with Master: d936ab250f71410aa88c0e9f4e9aaa85 (127.11.253.195:34213)
I20260806 03:50:54.422644 6215 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.11.253.195:58563
I20260806 03:50:54.423625 12279 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskToVWRe/build/release/bin/kudu as pid 6645
I20260806 03:50:54.423712 12279 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785988004637995-12279-0/minicluster-data/ts-2/wal/instance
I20260806 03:50:54.423862 12279 external_mini_cluster.cc:1428] Setting key b5b12a8d92fde85a47230e49daa2c7b2
I20260806 03:50:54.424643 12279 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 03:50:54.428965 6277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:50:54.426104 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33028 (local address 127.11.253.254:43933)
0806 03:50:54.426177 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0806 03:50:54.426178 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 03:50:54.426256 (+ 78us) server_negotiation.cc:407] Connection header received
0806 03:50:54.426314 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:50:54.426314 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:50:54.426323 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:50:54.426375 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:50:54.426523 (+ 148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:50:54.426991 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:50:54.427745 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:50:54.427941 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:50:54.428007 (+ 66us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:50:54.428710 (+ 703us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:50:54.428711 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:50:54.428711 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 03:50:54.428714 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:50:54.428724 (+ 10us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 03:50:54.428727 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:50:54.428789 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0806 03:50:54.428899 (+ 110us) server_negotiation.cc:299] Negotiation successful
0806 03:50:54.428909 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260806 03:50:54.430776 6215 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33028:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 03:50:54.432390 6215 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 03:50:54.439393 6839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:50:54.435031 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.193:38625 (local address 127.0.0.1:42752)
0806 03:50:54.435178 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0806 03:50:54.435221 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 03:50:54.435290 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:50:54.435649 (+ 359us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:50:54.435653 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:50:54.435655 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 03:50:54.435726 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:50:54.435728 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:50:54.437060 (+ 1332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:50:54.437061 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:50:54.438290 (+ 1229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:50:54.438293 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:50:54.439088 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:50:54.439089 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:50:54.439164 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:50:54.439179 (+ 15us) client_negotiation.cc:769] Sending connection context
0806 03:50:54.439258 (+ 79us) client_negotiation.cc:240] Negotiation successful
0806 03:50:54.439312 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"mutex_wait_us":103,"thread_start_us":79,"threads_started":1}
I20260806 03:50:54.439405 6846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:50:54.435153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42752 (local address 127.11.253.193:38625)
0806 03:50:54.435447 (+ 294us) server_negotiation.cc:206] Beginning negotiation
0806 03:50:54.435452 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 03:50:54.435481 (+ 29us) server_negotiation.cc:407] Connection header received
0806 03:50:54.435524 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:50:54.435527 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:50:54.435544 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:50:54.435624 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 03:50:54.435813 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:50:54.436961 (+ 1148us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:50:54.438386 (+ 1425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:50:54.438983 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:50:54.439089 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:50:54.439190 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0806 03:50:54.439272 (+ 82us) server_negotiation.cc:299] Negotiation successful
0806 03:50:54.439312 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":253,"thread_start_us":107,"threads_started":1}
I20260806 03:50:54.439836 6845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:50:54.434823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.195:34213 (local address 127.0.0.1:44604)
0806 03:50:54.435178 (+ 355us) negotiation.cc:107] Waiting for socket to connect
0806 03:50:54.435221 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 03:50:54.435289 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:50:54.436182 (+ 893us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:50:54.436183 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:50:54.436185 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 03:50:54.436221 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:50:54.436223 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:50:54.437045 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:50:54.437046 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:50:54.438509 (+ 1463us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:50:54.438511 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:50:54.439566 (+ 1055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:50:54.439567 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:50:54.439633 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:50:54.439638 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 03:50:54.439691 (+ 53us) client_negotiation.cc:240] Negotiation successful
0806 03:50:54.439743 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":318,"mutex_wait_us":98,"thread_start_us":67,"threads_started":1}
I20260806 03:50:54.439939 6838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:50:54.434823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.194:36055 (local address 127.0.0.1:35212)
0806 03:50:54.435179 (+ 356us) negotiation.cc:107] Waiting for socket to connect
0806 03:50:54.435221 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 03:50:54.435289 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:50:54.435902 (+ 613us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:50:54.435903 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:50:54.435905 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 03:50:54.435950 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:50:54.435952 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:50:54.437188 (+ 1236us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:50:54.437189 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:50:54.438509 (+ 1320us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:50:54.438511 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:50:54.439682 (+ 1171us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:50:54.439683 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:50:54.439735 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:50:54.439738 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 03:50:54.439783 (+ 45us) client_negotiation.cc:240] Negotiation successful
0806 03:50:54.439815 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":332,"mutex_wait_us":51,"thread_start_us":102,"threads_started":1}
I20260806 03:50:54.440091 6842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:50:54.434925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44604 (local address 127.11.253.195:34213)
0806 03:50:54.436031 (+ 1106us) server_negotiation.cc:206] Beginning negotiation
0806 03:50:54.436035 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 03:50:54.436044 (+ 9us) server_negotiation.cc:407] Connection header received
0806 03:50:54.436078 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:50:54.436079 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:50:54.436100 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:50:54.436184 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 03:50:54.436279 (+ 95us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:50:54.436956 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:50:54.438617 (+ 1661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:50:54.439358 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:50:54.439432 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:50:54.439521 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0806 03:50:54.439960 (+ 439us) server_negotiation.cc:299] Negotiation successful
0806 03:50:54.440004 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1063,"thread_start_us":108,"threads_started":1}
I20260806 03:50:54.440191 6383 tablet_service.cc:1467] Processing CreateTablet for tablet f1989e89887545118cd4700caeefdb5f (DEFAULT_TABLE table=test-table [id=8948b18ebb354a4ba68b40cc576977be]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 03:50:54.440521 6843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:50:54.435013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35212 (local address 127.11.253.194:36055)
0806 03:50:54.435749 (+ 736us) server_negotiation.cc:206] Beginning negotiation
0806 03:50:54.435754 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 03:50:54.435763 (+ 9us) server_negotiation.cc:407] Connection header received
0806 03:50:54.435805 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:50:54.435807 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:50:54.435818 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:50:54.435903 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 03:50:54.436048 (+ 145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:50:54.437110 (+ 1062us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:50:54.438624 (+ 1514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:50:54.439358 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:50:54.439432 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:50:54.439650 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0806 03:50:54.439878 (+ 228us) server_negotiation.cc:299] Negotiation successful
0806 03:50:54.440444 (+ 566us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":696,"thread_start_us":77,"threads_started":1}
I20260806 03:50:54.440618 6383 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f1989e89887545118cd4700caeefdb5f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:50:54.441256 6556 tablet_service.cc:1467] Processing CreateTablet for tablet f1989e89887545118cd4700caeefdb5f (DEFAULT_TABLE table=test-table [id=8948b18ebb354a4ba68b40cc576977be]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 03:50:54.441576 6556 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f1989e89887545118cd4700caeefdb5f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:50:54.443727 6847 tablet_bootstrap.cc:492] T f1989e89887545118cd4700caeefdb5f P 5950ce84fd61420d810ee844d602b2e3: Bootstrap starting.
I20260806 03:50:54.444296 6848 tablet_bootstrap.cc:492] T f1989e89887545118cd4700caeefdb5f P 1b860db760ba493fbc9ac415b613f536: Bootstrap starting.
I20260806 03:50:54.444418 6847 tablet_bootstrap.cc:654] T f1989e89887545118cd4700caeefdb5f P 5950ce84fd61420d810ee844d602b2e3: Neither blocks nor log segments found. Creating new log.
I20260806 03:50:54.444732 6847 log.cc:824] T f1989e89887545118cd4700caeefdb5f P 5950ce84fd61420d810ee844d602b2e3: Log is configured to *not* fsync() on all Append() calls
I20260806 03:50:54.444725 6738 tablet_service.cc:1467] Processing CreateTablet for tablet f1989e89887545118cd4700caeefdb5f (DEFAULT_TABLE table=test-table [id=8948b18ebb354a4ba68b40cc576977be]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 03:50:54.445071 6738 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f1989e89887545118cd4700caeefdb5f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:50:54.445287 6848 tablet_bootstrap.cc:654] T f1989e89887545118cd4700caeefdb5f P 1b860db760ba493fbc9ac415b613f536: Neither blocks nor log segments found. Creating new log.
I20260806 03:50:54.445609 6848 log.cc:824] T f1989e89887545118cd4700caeefdb5f P 1b860db760ba493fbc9ac415b613f536: Log is configured to *not* fsync() on all Append() calls
I20260806 03:50:54.446452 6847 tablet_bootstrap.cc:492] T f1989e89887545118cd4700caeefdb5f P 5950ce84fd61420d810ee844d602b2e3: No bootstrap required, opened a new log
I20260806 03:50:54.446513 6848 tablet_bootstrap.cc:492] T f1989e89887545118cd4700caeefdb5f P 1b860db760ba493fbc9ac415b613f536: No bootstrap required, opened a new log
I20260806 03:50:54.446548 6847 ts_tablet_manager.cc:1397] T f1989e89887545118cd4700caeefdb5f P 5950ce84fd61420d810ee844d602b2e3: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260806 03:50:54.446571 6848 ts_tablet_manager.cc:1397] T f1989e89887545118cd4700caeefdb5f P 1b860db760ba493fbc9ac415b613f536: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 03:50:54.448939 6847 raft_consensus.cc:348] T f1989e89887545118cd4700caeefdb5f P 5950ce84fd61420d810ee844d602b2e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b860db760ba493fbc9ac415b613f536" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 36055 } } peers { permanent_uuid: "d936ab250f71410aa88c0e9f4e9aaa85" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 34213 } } peers { permanent_uuid: "5950ce84fd61420d810ee844d602b2e3" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 38625 } }
I20260806 03:50:54.449103 6847 raft_consensus.cc:374] T f1989e89887545118cd4700caeefdb5f P 5950ce84fd61420d810ee844d602b2e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:50:54.449153 6847 raft_consensus.cc:729] T f1989e89887545118cd4700caeefdb5f P 5950ce84fd61420d810ee844d602b2e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5950ce84fd61420d810ee844d602b2e3, State: Initialized, Role: FOLLOWER
I20260806 03:50:54.449184 6848 raft_consensus.cc:348] T f1989e89887545118cd4700caeefdb5f P 1b860db760ba493fbc9ac415b613f536 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b860db760ba493fbc9ac415b613f536" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 36055 } } peers { permanent_uuid: "d936ab250f71410aa88c0e9f4e9aaa85" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 34213 } } peers { permanent_uuid: "5950ce84fd61420d810ee844d602b2e3" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 38625 } }
I20260806 03:50:54.449313 6847 consensus_queue.cc:260] T f1989e89887545118cd4700caeefdb5f P 5950ce84fd61420d810ee844d602b2e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b860db760ba493fbc9ac415b613f536" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 36055 } } peers { permanent_uuid: "d936ab250f71410aa88c0e9f4e9aaa85" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 34213 } } peers { permanent_uuid: "5950ce84fd61420d810ee844d602b2e3" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 38625 } }
I20260806 03:50:54.449345 6848 raft_consensus.cc:374] T f1989e89887545118cd4700caeefdb5f P 1b860db760ba493fbc9ac415b613f536 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:50:54.449401 6848 raft_consensus.cc:729] T f1989e89887545118cd4700caeefdb5f P 1b860db760ba493fbc9ac415b613f536 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b860db760ba493fbc9ac415b613f536, State: Initialized, Role: FOLLOWER
I20260806 03:50:54.449512 6848 consensus_queue.cc:260] T f1989e89887545118cd4700caeefdb5f P 1b860db760ba493fbc9ac415b613f536 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b860db760ba493fbc9ac415b613f536" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 36055 } } peers { permanent_uuid: "d936ab250f71410aa88c0e9f4e9aaa85" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 34213 } } peers { permanent_uuid: "5950ce84fd61420d810ee844d602b2e3" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 38625 } }
I20260806 03:50:54.449604 6847 ts_tablet_manager.cc:1428] T f1989e89887545118cd4700caeefdb5f P 5950ce84fd61420d810ee844d602b2e3: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 03:50:54.449657 6466 heartbeater.cc:503] Master 127.11.253.254:43933 was elected leader, sending a full tablet report...
I20260806 03:50:54.449805 6848 ts_tablet_manager.cc:1428] T f1989e89887545118cd4700caeefdb5f P 1b860db760ba493fbc9ac415b613f536: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260806 03:50:54.449962 6641 heartbeater.cc:503] Master 127.11.253.254:43933 was elected leader, sending a full tablet report...
I20260806 03:50:54.450029 6852 tablet_bootstrap.cc:492] T f1989e89887545118cd4700caeefdb5f P d936ab250f71410aa88c0e9f4e9aaa85: Bootstrap starting.
I20260806 03:50:54.450747 6852 tablet_bootstrap.cc:654] T f1989e89887545118cd4700caeefdb5f P d936ab250f71410aa88c0e9f4e9aaa85: Neither blocks nor log segments found. Creating new log.
I20260806 03:50:54.451122 6852 log.cc:824] T f1989e89887545118cd4700caeefdb5f P d936ab250f71410aa88c0e9f4e9aaa85: Log is configured to *not* fsync() on all Append() calls
I20260806 03:50:54.451792 6852 tablet_bootstrap.cc:492] T f1989e89887545118cd4700caeefdb5f P d936ab250f71410aa88c0e9f4e9aaa85: No bootstrap required, opened a new log
I20260806 03:50:54.451865 6852 ts_tablet_manager.cc:1397] T f1989e89887545118cd4700caeefdb5f P d936ab250f71410aa88c0e9f4e9aaa85: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 03:50:54.453792 6852 raft_consensus.cc:348] T f1989e89887545118cd4700caeefdb5f P d936ab250f71410aa88c0e9f4e9aaa85 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b860db760ba493fbc9ac415b613f536" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 36055 } } peers { permanent_uuid: "d936ab250f71410aa88c0e9f4e9aaa85" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 34213 } } peers { permanent_uuid: "5950ce84fd61420d810ee844d602b2e3" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 38625 } }
I20260806 03:50:54.453985 6852 raft_consensus.cc:374] T f1989e89887545118cd4700caeefdb5f P d936ab250f71410aa88c0e9f4e9aaa85 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:50:54.454012 6852 raft_consensus.cc:729] T f1989e89887545118cd4700caeefdb5f P d936ab250f71410aa88c0e9f4e9aaa85 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d936ab250f71410aa88c0e9f4e9aaa85, State: Initialized, Role: FOLLOWER
I20260806 03:50:54.454097 6852 consensus_queue.cc:260] T f1989e89887545118cd4700caeefdb5f P d936ab250f71410aa88c0e9f4e9aaa85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b860db760ba493fbc9ac415b613f536" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 36055 } } peers { permanent_uuid: "d936ab250f71410aa88c0e9f4e9aaa85" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 34213 } } peers { permanent_uuid: "5950ce84fd61420d810ee844d602b2e3" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 38625 } }
I20260806 03:50:54.454288 6852 ts_tablet_manager.cc:1428] T f1989e89887545118cd4700caeefdb5f P d936ab250f71410aa88c0e9f4e9aaa85: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260806 03:50:54.454406 6826 heartbeater.cc:503] Master 127.11.253.254:43933 was elected leader, sending a full tablet report...
I20260806 03:50:54.509976 6859 raft_consensus.cc:482] T f1989e89887545118cd4700caeefdb5f P d936ab250f71410aa88c0e9f4e9aaa85 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 03:50:54.510116 6859 raft_consensus.cc:504] T f1989e89887545118cd4700caeefdb5f P d936ab250f71410aa88c0e9f4e9aaa85 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b860db760ba493fbc9ac415b613f536" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 36055 } } peers { permanent_uuid: "d936ab250f71410aa88c0e9f4e9aaa85" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 34213 } } peers { permanent_uuid: "5950ce84fd61420d810ee844d602b2e3" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 38625 } }
I20260806 03:50:54.510612 6859 leader_election.cc:288] T f1989e89887545118cd4700caeefdb5f P d936ab250f71410aa88c0e9f4e9aaa85 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1b860db760ba493fbc9ac415b613f536 (127.11.253.194:36055), 5950ce84fd61420d810ee844d602b2e3 (127.11.253.193:38625)
I20260806 03:50:54.514568 6843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:50:54.510916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.253.195:37647 (local address 127.11.253.194:36055)
0806 03:50:54.511031 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0806 03:50:54.511033 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:50:54.511240 (+ 207us) server_negotiation.cc:407] Connection header received
0806 03:50:54.511270 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:50:54.511271 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:50:54.511284 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:50:54.511335 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 03:50:54.511463 (+ 128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:50:54.512156 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:50:54.513602 (+ 1446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:50:54.514177 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:50:54.514254 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:50:54.514263 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0806 03:50:54.514451 (+ 188us) server_negotiation.cc:299] Negotiation successful
0806 03:50:54.514493 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260806 03:50:54.514568 6846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:50:54.510861 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.253.195:44251 (local address 127.11.253.193:38625)
0806 03:50:54.510909 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0806 03:50:54.510911 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:50:54.510923 (+ 12us) server_negotiation.cc:407] Connection header received
0806 03:50:54.510980 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:50:54.510981 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:50:54.510995 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:50:54.511056 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 03:50:54.511257 (+ 201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:50:54.512019 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:50:54.513603 (+ 1584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:50:54.514177 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:50:54.514254 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:50:54.514263 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0806 03:50:54.514451 (+ 188us) server_negotiation.cc:299] Negotiation successful
0806 03:50:54.514495 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260806 03:50:54.514583 6830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:50:54.510785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.193:38625 (local address 127.11.253.195:44251)
0806 03:50:54.510855 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0806 03:50:54.510868 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 03:50:54.510911 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:50:54.511080 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:50:54.511081 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:50:54.511083 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 03:50:54.511164 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:50:54.511168 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:50:54.512114 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:50:54.512115 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:50:54.513494 (+ 1379us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:50:54.513498 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:50:54.514265 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:50:54.514266 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:50:54.514372 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:50:54.514375 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 03:50:54.514429 (+ 54us) client_negotiation.cc:240] Negotiation successful
0806 03:50:54.514468 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50,"mutex_wait_us":54}
I20260806 03:50:54.514597 6860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:50:54.510794 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.194:36055 (local address 127.11.253.195:37647)
0806 03:50:54.511142 (+ 348us) negotiation.cc:107] Waiting for socket to connect
0806 03:50:54.511153 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 03:50:54.511214 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:50:54.511344 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:50:54.511345 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:50:54.511346 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 03:50:54.511394 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:50:54.511395 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:50:54.512235 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:50:54.512235 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:50:54.513517 (+ 1282us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:50:54.513519 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:50:54.514265 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:50:54.514266 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:50:54.514349 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:50:54.514354 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 03:50:54.514420 (+ 66us) client_negotiation.cc:240] Negotiation successful
0806 03:50:54.514466 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":329,"mutex_wait_us":239,"thread_start_us":131,"threads_started":1}
I20260806 03:50:54.515038 6408 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f1989e89887545118cd4700caeefdb5f" candidate_uuid: "d936ab250f71410aa88c0e9f4e9aaa85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5950ce84fd61420d810ee844d602b2e3" is_pre_election: true
I20260806 03:50:54.515220 6408 raft_consensus.cc:2454] T f1989e89887545118cd4700caeefdb5f P 5950ce84fd61420d810ee844d602b2e3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d936ab250f71410aa88c0e9f4e9aaa85 in term 0.
W20260806 03:50:54.515337 6643 tablet.cc:2369] T f1989e89887545118cd4700caeefdb5f P 1b860db760ba493fbc9ac415b613f536: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 03:50:54.515312 6583 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f1989e89887545118cd4700caeefdb5f" candidate_uuid: "d936ab250f71410aa88c0e9f4e9aaa85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b860db760ba493fbc9ac415b613f536" is_pre_election: true
I20260806 03:50:54.515487 6583 raft_consensus.cc:2454] T f1989e89887545118cd4700caeefdb5f P 1b860db760ba493fbc9ac415b613f536 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d936ab250f71410aa88c0e9f4e9aaa85 in term 0.
I20260806 03:50:54.515460 6667 leader_election.cc:302] T f1989e89887545118cd4700caeefdb5f P d936ab250f71410aa88c0e9f4e9aaa85 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5950ce84fd61420d810ee844d602b2e3, d936ab250f71410aa88c0e9f4e9aaa85; no voters:
I20260806 03:50:54.515586 6859 raft_consensus.cc:2781] T f1989e89887545118cd4700caeefdb5f P d936ab250f71410aa88c0e9f4e9aaa85 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 03:50:54.515650 6859 raft_consensus.cc:482] T f1989e89887545118cd4700caeefdb5f P d936ab250f71410aa88c0e9f4e9aaa85 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 03:50:54.515676 6859 raft_consensus.cc:3037] T f1989e89887545118cd4700caeefdb5f P d936ab250f71410aa88c0e9f4e9aaa85 [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:50:54.516405 6859 raft_consensus.cc:504] T f1989e89887545118cd4700caeefdb5f P d936ab250f71410aa88c0e9f4e9aaa85 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b860db760ba493fbc9ac415b613f536" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 36055 } } peers { permanent_uuid: "d936ab250f71410aa88c0e9f4e9aaa85" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 34213 } } peers { permanent_uuid: "5950ce84fd61420d810ee844d602b2e3" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 38625 } }
I20260806 03:50:54.516558 6859 leader_election.cc:288] T f1989e89887545118cd4700caeefdb5f P d936ab250f71410aa88c0e9f4e9aaa85 [CANDIDATE]: Term 1 election: Requested vote from peers 1b860db760ba493fbc9ac415b613f536 (127.11.253.194:36055), 5950ce84fd61420d810ee844d602b2e3 (127.11.253.193:38625)
I20260806 03:50:54.516736 6583 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f1989e89887545118cd4700caeefdb5f" candidate_uuid: "d936ab250f71410aa88c0e9f4e9aaa85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b860db760ba493fbc9ac415b613f536"
I20260806 03:50:54.516759 6408 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f1989e89887545118cd4700caeefdb5f" candidate_uuid: "d936ab250f71410aa88c0e9f4e9aaa85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5950ce84fd61420d810ee844d602b2e3"
I20260806 03:50:54.516844 6408 raft_consensus.cc:3037] T f1989e89887545118cd4700caeefdb5f P 5950ce84fd61420d810ee844d602b2e3 [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:50:54.516844 6583 raft_consensus.cc:3037] T f1989e89887545118cd4700caeefdb5f P 1b860db760ba493fbc9ac415b613f536 [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:50:54.517849 6583 raft_consensus.cc:2454] T f1989e89887545118cd4700caeefdb5f P 1b860db760ba493fbc9ac415b613f536 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d936ab250f71410aa88c0e9f4e9aaa85 in term 1.
I20260806 03:50:54.517910 6408 raft_consensus.cc:2454] T f1989e89887545118cd4700caeefdb5f P 5950ce84fd61420d810ee844d602b2e3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d936ab250f71410aa88c0e9f4e9aaa85 in term 1.
I20260806 03:50:54.518114 6663 leader_election.cc:302] T f1989e89887545118cd4700caeefdb5f P d936ab250f71410aa88c0e9f4e9aaa85 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b860db760ba493fbc9ac415b613f536, d936ab250f71410aa88c0e9f4e9aaa85; no voters:
I20260806 03:50:54.518236 6859 raft_consensus.cc:2781] T f1989e89887545118cd4700caeefdb5f P d936ab250f71410aa88c0e9f4e9aaa85 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 03:50:54.518430 6859 raft_consensus.cc:686] T f1989e89887545118cd4700caeefdb5f P d936ab250f71410aa88c0e9f4e9aaa85 [term 1 LEADER]: Becoming Leader. State: Replica: d936ab250f71410aa88c0e9f4e9aaa85, State: Running, Role: LEADER
I20260806 03:50:54.518549 6859 consensus_queue.cc:237] T f1989e89887545118cd4700caeefdb5f P d936ab250f71410aa88c0e9f4e9aaa85 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b860db760ba493fbc9ac415b613f536" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 36055 } } peers { permanent_uuid: "d936ab250f71410aa88c0e9f4e9aaa85" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 34213 } } peers { permanent_uuid: "5950ce84fd61420d810ee844d602b2e3" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 38625 } }
I20260806 03:50:54.519490 6214 catalog_manager.cc:5495] T f1989e89887545118cd4700caeefdb5f P d936ab250f71410aa88c0e9f4e9aaa85 reported cstate change: term changed from 0 to 1, leader changed from <none> to d936ab250f71410aa88c0e9f4e9aaa85 (127.11.253.195). New cstate: current_term: 1 leader_uuid: "d936ab250f71410aa88c0e9f4e9aaa85" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b860db760ba493fbc9ac415b613f536" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 36055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d936ab250f71410aa88c0e9f4e9aaa85" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 34213 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5950ce84fd61420d810ee844d602b2e3" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 38625 } health_report { overall_health: UNKNOWN } } }
I20260806 03:50:54.534896 6842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:50:54.532313 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44614 (local address 127.11.253.195:34213)
0806 03:50:54.532382 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0806 03:50:54.532383 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 03:50:54.532392 (+ 9us) server_negotiation.cc:407] Connection header received
0806 03:50:54.532430 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:50:54.532431 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:50:54.532452 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:50:54.532518 (+ 66us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 03:50:54.532699 (+ 181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:50:54.533180 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:50:54.534023 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:50:54.534210 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:50:54.534267 (+ 57us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:50:54.534511 (+ 244us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 03:50:54.534608 (+ 97us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 03:50:54.534687 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0806 03:50:54.534738 (+ 51us) server_negotiation.cc:299] Negotiation successful
0806 03:50:54.534746 (+ 8us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260806 03:50:54.537109 6583 raft_consensus.cc:1261] T f1989e89887545118cd4700caeefdb5f P 1b860db760ba493fbc9ac415b613f536 [term 1 FOLLOWER]: Refusing update from remote peer d936ab250f71410aa88c0e9f4e9aaa85: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 03:50:54.537109 6408 raft_consensus.cc:1261] T f1989e89887545118cd4700caeefdb5f P 5950ce84fd61420d810ee844d602b2e3 [term 1 FOLLOWER]: Refusing update from remote peer d936ab250f71410aa88c0e9f4e9aaa85: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 03:50:54.537568 6859 consensus_queue.cc:1035] T f1989e89887545118cd4700caeefdb5f P d936ab250f71410aa88c0e9f4e9aaa85 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b860db760ba493fbc9ac415b613f536" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 36055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 03:50:54.537767 6861 consensus_queue.cc:1035] T f1989e89887545118cd4700caeefdb5f P d936ab250f71410aa88c0e9f4e9aaa85 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5950ce84fd61420d810ee844d602b2e3" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 38625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 03:50:54.540000 6864 mvcc.cc:205] Tried to move back new op lower bound from 7315407890580799488 to 7315407890508886016. Current Snapshot: MvccSnapshot[applied={T|T < 7315407890580799488}]
I20260806 03:50:54.540823 6865 mvcc.cc:205] Tried to move back new op lower bound from 7315407890580799488 to 7315407890508886016. Current Snapshot: MvccSnapshot[applied={T|T < 7315407890580799488 or (T in {7315407890580799488})}]
I20260806 03:50:54.542685 6214 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:33028:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 03:50:54.542812 6214 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:33028:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 03:50:54.543607 6214 catalog_manager.cc:5782] T 00000000000000000000000000000000 P fb96b8b0b07140128e93e2e701c08717: Sending DeleteTablet for 3 replicas of tablet f1989e89887545118cd4700caeefdb5f
I20260806 03:50:54.543947 6556 tablet_service.cc:1514] Processing DeleteTablet for tablet f1989e89887545118cd4700caeefdb5f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 03:50:54 UTC) from {username='slave'} at 127.0.0.1:35212
I20260806 03:50:54.544020 6738 tablet_service.cc:1514] Processing DeleteTablet for tablet f1989e89887545118cd4700caeefdb5f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 03:50:54 UTC) from {username='slave'} at 127.0.0.1:44604
I20260806 03:50:54.544215 6876 tablet_replica.cc:330] T f1989e89887545118cd4700caeefdb5f P 1b860db760ba493fbc9ac415b613f536: stopping tablet replica
I20260806 03:50:54.544230 6383 tablet_service.cc:1514] Processing DeleteTablet for tablet f1989e89887545118cd4700caeefdb5f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 03:50:54 UTC) from {username='slave'} at 127.0.0.1:42752
I20260806 03:50:54.544399 6876 raft_consensus.cc:2229] T f1989e89887545118cd4700caeefdb5f P 1b860db760ba493fbc9ac415b613f536 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 03:50:54.544505 6878 tablet_replica.cc:330] T f1989e89887545118cd4700caeefdb5f P 5950ce84fd61420d810ee844d602b2e3: stopping tablet replica
I20260806 03:50:54.544528 6876 raft_consensus.cc:2258] T f1989e89887545118cd4700caeefdb5f P 1b860db760ba493fbc9ac415b613f536 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 03:50:54.544665 6878 raft_consensus.cc:2229] T f1989e89887545118cd4700caeefdb5f P 5950ce84fd61420d810ee844d602b2e3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 03:50:54.544836 6878 raft_consensus.cc:2258] T f1989e89887545118cd4700caeefdb5f P 5950ce84fd61420d810ee844d602b2e3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 03:50:54.544836 12279 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskToVWRe/build/release/bin/kudu with pid 6303
I20260806 03:50:54.545460 6876 ts_tablet_manager.cc:1905] T f1989e89887545118cd4700caeefdb5f P 1b860db760ba493fbc9ac415b613f536: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 03:50:54.547945 6876 ts_tablet_manager.cc:1918] T f1989e89887545118cd4700caeefdb5f P 1b860db760ba493fbc9ac415b613f536: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 03:50:54.548060 6876 log.cc:1196] T f1989e89887545118cd4700caeefdb5f P 1b860db760ba493fbc9ac415b613f536: Deleting WAL directory at /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785988004637995-12279-0/minicluster-data/ts-1/wal/wals/f1989e89887545118cd4700caeefdb5f
I20260806 03:50:54.548377 6876 ts_tablet_manager.cc:1939] T f1989e89887545118cd4700caeefdb5f P 1b860db760ba493fbc9ac415b613f536: Deleting consensus metadata
I20260806 03:50:54.553556 6189 catalog_manager.cc:4841] TS 1b860db760ba493fbc9ac415b613f536 (127.11.253.194:36055): tablet f1989e89887545118cd4700caeefdb5f (table test-table [id=8948b18ebb354a4ba68b40cc576977be]) successfully deleted
I20260806 03:50:54.553992 6877 tablet_replica.cc:330] T f1989e89887545118cd4700caeefdb5f P d936ab250f71410aa88c0e9f4e9aaa85: stopping tablet replica
I20260806 03:50:54.554129 6877 raft_consensus.cc:2229] T f1989e89887545118cd4700caeefdb5f P d936ab250f71410aa88c0e9f4e9aaa85 [term 1 LEADER]: Raft consensus shutting down.
I20260806 03:50:54.554265 6877 raft_consensus.cc:2258] T f1989e89887545118cd4700caeefdb5f P d936ab250f71410aa88c0e9f4e9aaa85 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 03:50:54.554809 6877 ts_tablet_manager.cc:1905] T f1989e89887545118cd4700caeefdb5f P d936ab250f71410aa88c0e9f4e9aaa85: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 03:50:54.556965 6877 ts_tablet_manager.cc:1918] T f1989e89887545118cd4700caeefdb5f P d936ab250f71410aa88c0e9f4e9aaa85: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 03:50:54.557067 6877 log.cc:1196] T f1989e89887545118cd4700caeefdb5f P d936ab250f71410aa88c0e9f4e9aaa85: Deleting WAL directory at /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785988004637995-12279-0/minicluster-data/ts-2/wal/wals/f1989e89887545118cd4700caeefdb5f
I20260806 03:50:54.557385 6877 ts_tablet_manager.cc:1939] T f1989e89887545118cd4700caeefdb5f P d936ab250f71410aa88c0e9f4e9aaa85: Deleting consensus metadata
I20260806 03:50:54.557930 6192 catalog_manager.cc:4841] TS d936ab250f71410aa88c0e9f4e9aaa85 (127.11.253.195:34213): tablet f1989e89887545118cd4700caeefdb5f (table test-table [id=8948b18ebb354a4ba68b40cc576977be]) successfully deleted
W20260806 03:50:54.559358 6192 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.253.193:38625 (error 108)
I20260806 03:50:54.559669 12279 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskToVWRe/build/release/bin/kudu with pid 6469
I20260806 03:50:54.561843 6839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:50:54.559808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.193:38625 (local address 127.0.0.1:42766)
0806 03:50:54.561746 (+ 1938us) negotiation.cc:107] Waiting for socket to connect
0806 03:50:54.561810 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.253.193:38625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1909}
W20260806 03:50:54.562709 6192 catalog_manager.cc:4584] TS 5950ce84fd61420d810ee844d602b2e3 (127.11.253.193:38625): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet f1989e89887545118cd4700caeefdb5f: Network error: Client connection negotiation failed: client connection to 127.11.253.193:38625: connect: Connection refused (error 111)
I20260806 03:50:54.569315 12279 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskToVWRe/build/release/bin/kudu with pid 6645
I20260806 03:50:54.584177 12279 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskToVWRe/build/release/bin/kudu with pid 6168
2026-08-06T03:50:54Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (779 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785988004637995-12279-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 03:50:54 dist-test-slave-6ntk krb5kdc[6882](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 03:50:54 dist-test-slave-6ntk krb5kdc[6882](info): set up 2 sockets
Aug 06 03:50:54 dist-test-slave-6ntk krb5kdc[6882](info): commencing operation
krb5kdc: starting...
W20260806 03:50:55.801540 12279 mini_kdc.cc:121] Time spent starting KDC: real 1.191s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785988004637995-12279-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785988004637995-12279-0/krb5kdc/test-user.keytab.
Aug 06 03:50:55 dist-test-slave-6ntk krb5kdc[6882](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988255, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T03:50:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T03:50:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.253.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.253.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.253.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.254.keytab.
Entry for principal kudu/127.11.253.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.254.keytab.
I20260806 03:50:55.933584 12279 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskToVWRe/build/release/bin/kudu
/tmp/dist-test-taskToVWRe/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785988004637995-12279-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785988004637995-12279-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785988004637995-12279-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785988004637995-12279-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.253.254:35999
--webserver_interface=127.11.253.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.254.keytab
--principal=kudu/127.11.253.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.253.212:35221
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.253.254:35999
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785988004637995-12279-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785988004637995-12279-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785988004637995-12279-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:50:56.020961 6901 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:50:56.021188 6901 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:50:56.021211 6901 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:50:56.022414 6901 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 03:50:56.022460 6901 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:50:56.022475 6901 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:50:56.022487 6901 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 03:50:56.022500 6901 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 03:50:56.023964 6901 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.253.212:35221
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785988004637995-12279-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785988004637995-12279-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.253.254:35999
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.254.keytab
--principal=kudu/127.11.253.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.253.254:35999
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785988004637995-12279-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.253.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6901
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785988004637995-12279-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:50:56.024173 6901 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:50:56.024355 6901 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:50:56.029596 6914 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:50:56.029673 6901 server_base.cc:1034] running on GCE node
W20260806 03:50:56.029635 6912 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:50:56.029635 6909 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:50:56.030177 6901 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:50:56.030512 6901 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:50:56.031694 6901 hybrid_clock.cc:648] HybridClock initialized: now 1785988256031670 us; error 36 us; skew 500 ppm
Aug 06 03:50:56 dist-test-slave-6ntk krb5kdc[6882](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988256, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.253.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:50:56.034425 6901 init.cc:377] Logged in from keytab as kudu/127.11.253.254@KRBTEST.COM (short username kudu)
I20260806 03:50:56.034901 6901 webserver.cc:460] Webserver started at http://127.11.253.254:36291/ using document root <none> and password file <none>
I20260806 03:50:56.035140 6901 fs_manager.cc:359] Metadata directory not provided
I20260806 03:50:56.035210 6901 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:50:56.035374 6901 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:50:56.036490 6901 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785988004637995-12279-0/minicluster-data/master-0/data/instance:
uuid: "0a24df93ed8446a7ad7ea54a614d1ef3"
format_stamp: "Formatted at 2026-08-06 03:50:56 on dist-test-slave-6ntk"
server_key: "cffc2b71227283544ed7ad4feec3cac4"
server_key_iv: "a6317bf59bc978a398cae79e39ad7d83"
server_key_version: "encryptionkey@0"
I20260806 03:50:56.036913 6901 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785988004637995-12279-0/minicluster-data/master-0/wal/instance:
uuid: "0a24df93ed8446a7ad7ea54a614d1ef3"
format_stamp: "Formatted at 2026-08-06 03:50:56 on dist-test-slave-6ntk"
server_key: "cffc2b71227283544ed7ad4feec3cac4"
server_key_iv: "a6317bf59bc978a398cae79e39ad7d83"
server_key_version: "encryptionkey@0"
I20260806 03:50:56.038350 6901 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 03:50:56.039137 6924 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:50:56.039320 6901 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 03:50:56.039436 6901 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785988004637995-12279-0/minicluster-data/master-0/data,/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785988004637995-12279-0/minicluster-data/master-0/wal
uuid: "0a24df93ed8446a7ad7ea54a614d1ef3"
format_stamp: "Formatted at 2026-08-06 03:50:56 on dist-test-slave-6ntk"
server_key: "cffc2b71227283544ed7ad4feec3cac4"
server_key_iv: "a6317bf59bc978a398cae79e39ad7d83"
server_key_version: "encryptionkey@0"
I20260806 03:50:56.039515 6901 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785988004637995-12279-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785988004637995-12279-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785988004637995-12279-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:50:56.054494 6901 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:50:56.056962 6901 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:50:56.057137 6901 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:50:56.061779 6901 rpc_server.cc:307] RPC server started. Bound to: 127.11.253.254:35999
I20260806 03:50:56.061825 6995 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.253.254:35999 every 8 connection(s)
I20260806 03:50:56.062312 6901 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785988004637995-12279-0/minicluster-data/master-0/data/info.pb
I20260806 03:50:56.063298 6996 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:50:56.065619 6996 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0a24df93ed8446a7ad7ea54a614d1ef3: Bootstrap starting.
I20260806 03:50:56.066309 6996 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0a24df93ed8446a7ad7ea54a614d1ef3: Neither blocks nor log segments found. Creating new log.
I20260806 03:50:56.066568 6996 log.cc:824] T 00000000000000000000000000000000 P 0a24df93ed8446a7ad7ea54a614d1ef3: Log is configured to *not* fsync() on all Append() calls
I20260806 03:50:56.067250 6996 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0a24df93ed8446a7ad7ea54a614d1ef3: No bootstrap required, opened a new log
I20260806 03:50:56.068843 6996 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0a24df93ed8446a7ad7ea54a614d1ef3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a24df93ed8446a7ad7ea54a614d1ef3" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 35999 } }
I20260806 03:50:56.068985 6996 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0a24df93ed8446a7ad7ea54a614d1ef3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:50:56.069010 6996 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0a24df93ed8446a7ad7ea54a614d1ef3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a24df93ed8446a7ad7ea54a614d1ef3, State: Initialized, Role: FOLLOWER
I20260806 03:50:56.069161 6996 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0a24df93ed8446a7ad7ea54a614d1ef3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a24df93ed8446a7ad7ea54a614d1ef3" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 35999 } }
I20260806 03:50:56.069252 6996 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0a24df93ed8446a7ad7ea54a614d1ef3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 03:50:56.069278 6996 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0a24df93ed8446a7ad7ea54a614d1ef3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 03:50:56.069306 6996 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0a24df93ed8446a7ad7ea54a614d1ef3 [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:50:56.069811 6996 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0a24df93ed8446a7ad7ea54a614d1ef3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a24df93ed8446a7ad7ea54a614d1ef3" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 35999 } }
I20260806 03:50:56.069973 6996 leader_election.cc:302] T 00000000000000000000000000000000 P 0a24df93ed8446a7ad7ea54a614d1ef3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0a24df93ed8446a7ad7ea54a614d1ef3; no voters:
I20260806 03:50:56.070056 12279 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskToVWRe/build/release/bin/kudu as pid 6901
I20260806 03:50:56.070170 6996 leader_election.cc:288] T 00000000000000000000000000000000 P 0a24df93ed8446a7ad7ea54a614d1ef3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 03:50:56.070191 12279 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785988004637995-12279-0/minicluster-data/master-0/wal/instance
I20260806 03:50:56.070255 7000 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0a24df93ed8446a7ad7ea54a614d1ef3 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 03:50:56.070415 12279 external_mini_cluster.cc:1428] Setting key e5d6015b0858a97e64fd8765c4e9e0ee
I20260806 03:50:56.070417 7000 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0a24df93ed8446a7ad7ea54a614d1ef3 [term 1 LEADER]: Becoming Leader. State: Replica: 0a24df93ed8446a7ad7ea54a614d1ef3, State: Running, Role: LEADER
I20260806 03:50:56.070469 6996 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0a24df93ed8446a7ad7ea54a614d1ef3 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 03:50:56.070509 7000 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0a24df93ed8446a7ad7ea54a614d1ef3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a24df93ed8446a7ad7ea54a614d1ef3" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 35999 } }
I20260806 03:50:56.070957 7002 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0a24df93ed8446a7ad7ea54a614d1ef3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0a24df93ed8446a7ad7ea54a614d1ef3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a24df93ed8446a7ad7ea54a614d1ef3" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 35999 } } }
I20260806 03:50:56.071049 7002 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0a24df93ed8446a7ad7ea54a614d1ef3 [sys.catalog]: This master's current role is: LEADER
I20260806 03:50:56.071202 7003 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0a24df93ed8446a7ad7ea54a614d1ef3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0a24df93ed8446a7ad7ea54a614d1ef3. Latest consensus state: current_term: 1 leader_uuid: "0a24df93ed8446a7ad7ea54a614d1ef3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a24df93ed8446a7ad7ea54a614d1ef3" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 35999 } } }
I20260806 03:50:56.071341 7003 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0a24df93ed8446a7ad7ea54a614d1ef3 [sys.catalog]: This master's current role is: LEADER
W20260806 03:50:56.072512 7028 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0a24df93ed8446a7ad7ea54a614d1ef3: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 03:50:56.072616 7028 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 03:50:56.072739 7014 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 03:50:56.072949 7014 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 03:50:56.074672 7014 catalog_manager.cc:1347] Generated new cluster ID: ec094ad38a5d454a965e62750a27c83b
I20260806 03:50:56.074729 7014 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 03:50:56 dist-test-slave-6ntk krb5kdc[6882](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988255, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.253.254@KRBTEST.COM
I20260806 03:50:56.082785 7012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:50:56.071092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54820 (local address 127.11.253.254:35999)
0806 03:50:56.071388 (+ 296us) server_negotiation.cc:206] Beginning negotiation
0806 03:50:56.071400 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 03:50:56.071459 (+ 59us) server_negotiation.cc:407] Connection header received
0806 03:50:56.071594 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:50:56.071602 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:50:56.071673 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:50:56.071834 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:50:56.072688 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:50:56.074061 (+ 1373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:50:56.075133 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:50:56.075836 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:50:56.075959 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:50:56.079001 (+ 3042us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:50:56.079005 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:50:56.079006 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:50:56.079019 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:50:56.081018 (+ 1999us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:50:56.081456 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:50:56.081460 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:50:56.081460 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:50:56.081577 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:50:56.081810 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:50:56.081810 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:50:56.081811 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:50:56.082132 (+ 321us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:50:56.082292 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0806 03:50:56.082408 (+ 116us) server_negotiation.cc:299] Negotiation successful
0806 03:50:56.082541 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":249,"thread_start_us":73,"threads_started":1}
I20260806 03:50:56.086526 7014 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 03:50:56.087188 7014 catalog_manager.cc:1504] Loading token signing keys...
I20260806 03:50:56.088949 7014 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0a24df93ed8446a7ad7ea54a614d1ef3: Generated new TSK 0
I20260806 03:50:56.089111 7014 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.253.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.253.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.253.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.193.keytab.
Entry for principal kudu/127.11.253.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.193.keytab.
I20260806 03:50:56.121484 12279 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskToVWRe/build/release/bin/kudu
/tmp/dist-test-taskToVWRe/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785988004637995-12279-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785988004637995-12279-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785988004637995-12279-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785988004637995-12279-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.253.193:0
--local_ip_for_outbound_sockets=127.11.253.193
--webserver_interface=127.11.253.193
--webserver_port=0
--tserver_master_addrs=127.11.253.254:35999
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.193.keytab
--principal=kudu/127.11.253.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.253.212:35221
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785988004637995-12279-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785988004637995-12279-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785988004637995-12279-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:50:56.210825 7036 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:50:56.211054 7036 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:50:56.211078 7036 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:50:56.212302 7036 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:50:56.212350 7036 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:50:56.212378 7036 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.253.193
I20260806 03:50:56.213923 7036 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.253.212:35221
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785988004637995-12279-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785988004637995-12279-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.193.keytab
--principal=kudu/127.11.253.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.253.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785988004637995-12279-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.253.193
--webserver_port=0
--tserver_master_addrs=127.11.253.254:35999
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7036
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.253.193
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785988004637995-12279-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:50:56.214195 7036 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:50:56.214447 7036 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:50:56.219707 7043 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:50:56.219704 7046 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:50:56.219786 7044 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:50:56.219949 7036 server_base.cc:1034] running on GCE node
I20260806 03:50:56.220121 7036 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:50:56.220392 7036 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:50:56.221554 7036 hybrid_clock.cc:648] HybridClock initialized: now 1785988256221542 us; error 31 us; skew 500 ppm
Aug 06 03:50:56 dist-test-slave-6ntk krb5kdc[6882](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988256, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.253.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:50:56.224370 7036 init.cc:377] Logged in from keytab as kudu/127.11.253.193@KRBTEST.COM (short username kudu)
I20260806 03:50:56.224833 7036 webserver.cc:460] Webserver started at http://127.11.253.193:36593/ using document root <none> and password file <none>
I20260806 03:50:56.225093 7036 fs_manager.cc:359] Metadata directory not provided
I20260806 03:50:56.225140 7036 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:50:56.225278 7036 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:50:56.226586 7036 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785988004637995-12279-0/minicluster-data/ts-0/data/instance:
uuid: "ded1f9cb216243e4be42c68100817ef3"
format_stamp: "Formatted at 2026-08-06 03:50:56 on dist-test-slave-6ntk"
server_key: "ff502850be38b09878b47b3cd4a92886"
server_key_iv: "2233262dcb658a862899d3ee717845d7"
server_key_version: "encryptionkey@0"
I20260806 03:50:56.226962 7036 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785988004637995-12279-0/minicluster-data/ts-0/wal/instance:
uuid: "ded1f9cb216243e4be42c68100817ef3"
format_stamp: "Formatted at 2026-08-06 03:50:56 on dist-test-slave-6ntk"
server_key: "ff502850be38b09878b47b3cd4a92886"
server_key_iv: "2233262dcb658a862899d3ee717845d7"
server_key_version: "encryptionkey@0"
I20260806 03:50:56.228384 7036 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 03:50:56.229174 7055 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:50:56.229452 7036 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 03:50:56.229542 7036 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785988004637995-12279-0/minicluster-data/ts-0/data,/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785988004637995-12279-0/minicluster-data/ts-0/wal
uuid: "ded1f9cb216243e4be42c68100817ef3"
format_stamp: "Formatted at 2026-08-06 03:50:56 on dist-test-slave-6ntk"
server_key: "ff502850be38b09878b47b3cd4a92886"
server_key_iv: "2233262dcb658a862899d3ee717845d7"
server_key_version: "encryptionkey@0"
I20260806 03:50:56.229599 7036 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785988004637995-12279-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785988004637995-12279-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785988004637995-12279-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:50:56.240068 7036 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:50:56.242130 7036 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:50:56.242256 7036 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:50:56.242494 7036 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 03:50:56.242777 7036 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 03:50:56.242815 7036 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 03:50:56.242838 7036 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 03:50:56.242851 7036 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 03:50:56.248581 7036 rpc_server.cc:307] RPC server started. Bound to: 127.11.253.193:41781
I20260806 03:50:56.248663 7207 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.253.193:41781 every 8 connection(s)
I20260806 03:50:56.248982 7036 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785988004637995-12279-0/minicluster-data/ts-0/data/info.pb
Aug 06 03:50:56 dist-test-slave-6ntk krb5kdc[6882](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988256, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.253.193@KRBTEST.COM for kudu/127.11.253.254@KRBTEST.COM
I20260806 03:50:56.258080 12279 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskToVWRe/build/release/bin/kudu as pid 7036
I20260806 03:50:56.258246 12279 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785988004637995-12279-0/minicluster-data/ts-0/wal/instance
I20260806 03:50:56.258500 12279 external_mini_cluster.cc:1428] Setting key d57a027a94129ab2529e5116fe8302ac
I20260806 03:50:56.259018 7012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:50:56.249740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.253.193:37449 (local address 127.11.253.254:35999)
0806 03:50:56.249784 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0806 03:50:56.249786 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:50:56.250076 (+ 290us) server_negotiation.cc:407] Connection header received
0806 03:50:56.250655 (+ 579us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:50:56.250657 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:50:56.250673 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:50:56.250728 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:50:56.251708 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:50:56.252221 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:50:56.253263 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:50:56.253477 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:50:56.253556 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:50:56.255761 (+ 2205us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:50:56.255762 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:50:56.255762 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:50:56.255769 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:50:56.257227 (+ 1458us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:50:56.257634 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:50:56.257635 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:50:56.257635 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:50:56.257686 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:50:56.258176 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:50:56.258177 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:50:56.258178 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:50:56.258360 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:50:56.258450 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0806 03:50:56.258858 (+ 408us) server_negotiation.cc:299] Negotiation successful
0806 03:50:56.258953 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260806 03:50:56.259217 7211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:50:56.249705 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:35999 (local address 127.11.253.193:37449)
0806 03:50:56.249946 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0806 03:50:56.249992 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0806 03:50:56.250537 (+ 545us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:50:56.250846 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:50:56.250850 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:50:56.251421 (+ 571us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:50:56.251607 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:50:56.251611 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:50:56.252327 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:50:56.252328 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:50:56.253149 (+ 821us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:50:56.253151 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:50:56.253564 (+ 413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:50:56.253565 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:50:56.253694 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:50:56.254168 (+ 474us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:50:56.254178 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:50:56.255660 (+ 1482us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:50:56.257316 (+ 1656us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:50:56.257324 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:50:56.257326 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:50:56.257563 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:50:56.257911 (+ 348us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:50:56.257913 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:50:56.257913 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:50:56.258069 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:50:56.258451 (+ 382us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:50:56.258460 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:50:56.258639 (+ 179us) client_negotiation.cc:769] Sending connection context
0806 03:50:56.258759 (+ 120us) client_negotiation.cc:240] Negotiation successful
0806 03:50:56.258965 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":101,"threads_started":1}
I20260806 03:50:56.259768 7208 heartbeater.cc:347] Connected to a master server at 127.11.253.254:35999
I20260806 03:50:56.259884 7208 heartbeater.cc:464] Registering TS with master...
I20260806 03:50:56.260066 7208 heartbeater.cc:511] Master 127.11.253.254:35999 requested a full tablet report, sending...
I20260806 03:50:56.260639 6944 ts_manager.cc:194] Registered new tserver with Master: ded1f9cb216243e4be42c68100817ef3 (127.11.253.193:41781)
I20260806 03:50:56.261518 6944 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.253.193@KRBTEST.COM'} at 127.11.253.193:37449
WARNING: no policy specified for kudu/127.11.253.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.253.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.253.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.194.keytab.
Entry for principal kudu/127.11.253.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.194.keytab.
I20260806 03:50:56.289479 12279 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskToVWRe/build/release/bin/kudu
/tmp/dist-test-taskToVWRe/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785988004637995-12279-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785988004637995-12279-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785988004637995-12279-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785988004637995-12279-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.253.194:0
--local_ip_for_outbound_sockets=127.11.253.194
--webserver_interface=127.11.253.194
--webserver_port=0
--tserver_master_addrs=127.11.253.254:35999
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.194.keytab
--principal=kudu/127.11.253.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.253.212:35221
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785988004637995-12279-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785988004637995-12279-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785988004637995-12279-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:50:56.374328 7214 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:50:56.374575 7214 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:50:56.374614 7214 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:50:56.375756 7214 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:50:56.375820 7214 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:50:56.375865 7214 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.253.194
I20260806 03:50:56.377349 7214 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.253.212:35221
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785988004637995-12279-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785988004637995-12279-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.194.keytab
--principal=kudu/127.11.253.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.253.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785988004637995-12279-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.253.194
--webserver_port=0
--tserver_master_addrs=127.11.253.254:35999
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7214
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.253.194
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785988004637995-12279-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:50:56.377629 7214 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:50:56.377888 7214 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:50:56.382918 7226 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:50:56.382921 7222 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:50:56.383088 7214 server_base.cc:1034] running on GCE node
W20260806 03:50:56.382913 7219 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:50:56.383484 7214 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:50:56.383700 7214 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:50:56.384845 7214 hybrid_clock.cc:648] HybridClock initialized: now 1785988256384832 us; error 31 us; skew 500 ppm
Aug 06 03:50:56 dist-test-slave-6ntk krb5kdc[6882](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988256, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.253.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:50:56.387617 7214 init.cc:377] Logged in from keytab as kudu/127.11.253.194@KRBTEST.COM (short username kudu)
I20260806 03:50:56.388212 7214 webserver.cc:460] Webserver started at http://127.11.253.194:44345/ using document root <none> and password file <none>
I20260806 03:50:56.388489 7214 fs_manager.cc:359] Metadata directory not provided
I20260806 03:50:56.388536 7214 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:50:56.388681 7214 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:50:56.389726 7214 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785988004637995-12279-0/minicluster-data/ts-1/data/instance:
uuid: "6a613dea1c854b1993d8ffe2f4529640"
format_stamp: "Formatted at 2026-08-06 03:50:56 on dist-test-slave-6ntk"
server_key: "6e6812d56918cc20c8ff10358afcdc9b"
server_key_iv: "38921338f419e3dd9ba900ca530c4cd0"
server_key_version: "encryptionkey@0"
I20260806 03:50:56.390117 7214 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785988004637995-12279-0/minicluster-data/ts-1/wal/instance:
uuid: "6a613dea1c854b1993d8ffe2f4529640"
format_stamp: "Formatted at 2026-08-06 03:50:56 on dist-test-slave-6ntk"
server_key: "6e6812d56918cc20c8ff10358afcdc9b"
server_key_iv: "38921338f419e3dd9ba900ca530c4cd0"
server_key_version: "encryptionkey@0"
I20260806 03:50:56.391502 7214 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 03:50:56.392352 7233 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:50:56.392553 7214 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 03:50:56.392665 7214 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785988004637995-12279-0/minicluster-data/ts-1/data,/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785988004637995-12279-0/minicluster-data/ts-1/wal
uuid: "6a613dea1c854b1993d8ffe2f4529640"
format_stamp: "Formatted at 2026-08-06 03:50:56 on dist-test-slave-6ntk"
server_key: "6e6812d56918cc20c8ff10358afcdc9b"
server_key_iv: "38921338f419e3dd9ba900ca530c4cd0"
server_key_version: "encryptionkey@0"
I20260806 03:50:56.392740 7214 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785988004637995-12279-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785988004637995-12279-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785988004637995-12279-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:50:56.406641 7214 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:50:56.408998 7214 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:50:56.409168 7214 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:50:56.409403 7214 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 03:50:56.409718 7214 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 03:50:56.409771 7214 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 03:50:56.409807 7214 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 03:50:56.409835 7214 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 03:50:56.415732 7214 rpc_server.cc:307] RPC server started. Bound to: 127.11.253.194:43277
I20260806 03:50:56.415844 7380 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.253.194:43277 every 8 connection(s)
I20260806 03:50:56.416181 7214 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785988004637995-12279-0/minicluster-data/ts-1/data/info.pb
Aug 06 03:50:56 dist-test-slave-6ntk krb5kdc[6882](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988256, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.253.194@KRBTEST.COM for kudu/127.11.253.254@KRBTEST.COM
I20260806 03:50:56.425532 12279 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskToVWRe/build/release/bin/kudu as pid 7214
I20260806 03:50:56.425678 12279 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785988004637995-12279-0/minicluster-data/ts-1/wal/instance
I20260806 03:50:56.425976 12279 external_mini_cluster.cc:1428] Setting key 444238ff4332e60ae2d53a1fa0d6f6b1
I20260806 03:50:56.426272 7012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:50:56.416921 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.253.194:40685 (local address 127.11.253.254:35999)
0806 03:50:56.416984 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0806 03:50:56.416986 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:50:56.417177 (+ 191us) server_negotiation.cc:407] Connection header received
0806 03:50:56.417681 (+ 504us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:50:56.417682 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:50:56.417691 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:50:56.417763 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:50:56.418830 (+ 1067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:50:56.419354 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:50:56.420364 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:50:56.420555 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:50:56.420613 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:50:56.423021 (+ 2408us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:50:56.423022 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:50:56.423023 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:50:56.423031 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:50:56.424384 (+ 1353us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:50:56.424856 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:50:56.424857 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:50:56.424858 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:50:56.424920 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:50:56.425271 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:50:56.425272 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:50:56.425273 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:50:56.425449 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:50:56.425527 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0806 03:50:56.426044 (+ 517us) server_negotiation.cc:299] Negotiation successful
0806 03:50:56.426161 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260806 03:50:56.426481 7385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:50:56.416887 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:35999 (local address 127.11.253.194:40685)
0806 03:50:56.417047 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0806 03:50:56.417089 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 03:50:56.417590 (+ 501us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:50:56.417849 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:50:56.417852 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:50:56.418512 (+ 660us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:50:56.418708 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:50:56.418713 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:50:56.419445 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:50:56.419446 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:50:56.420269 (+ 823us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:50:56.420271 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:50:56.420632 (+ 361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:50:56.420633 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:50:56.420733 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:50:56.421318 (+ 585us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:50:56.421327 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:50:56.422916 (+ 1589us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:50:56.424492 (+ 1576us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:50:56.424499 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:50:56.424501 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:50:56.424761 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:50:56.424994 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:50:56.424995 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:50:56.424995 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:50:56.425112 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:50:56.425553 (+ 441us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:50:56.425560 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:50:56.425789 (+ 229us) client_negotiation.cc:769] Sending connection context
0806 03:50:56.426006 (+ 217us) client_negotiation.cc:240] Negotiation successful
0806 03:50:56.426218 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":88,"threads_started":1}
I20260806 03:50:56.427031 7381 heartbeater.cc:347] Connected to a master server at 127.11.253.254:35999
I20260806 03:50:56.427143 7381 heartbeater.cc:464] Registering TS with master...
I20260806 03:50:56.427367 7381 heartbeater.cc:511] Master 127.11.253.254:35999 requested a full tablet report, sending...
I20260806 03:50:56.427757 6944 ts_manager.cc:194] Registered new tserver with Master: 6a613dea1c854b1993d8ffe2f4529640 (127.11.253.194:43277)
I20260806 03:50:56.428164 6944 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.253.194@KRBTEST.COM'} at 127.11.253.194:40685
WARNING: no policy specified for kudu/127.11.253.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.253.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.253.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.195.keytab.
Entry for principal kudu/127.11.253.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.195.keytab.
I20260806 03:50:56.457734 12279 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskToVWRe/build/release/bin/kudu
/tmp/dist-test-taskToVWRe/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785988004637995-12279-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785988004637995-12279-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785988004637995-12279-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785988004637995-12279-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.253.195:0
--local_ip_for_outbound_sockets=127.11.253.195
--webserver_interface=127.11.253.195
--webserver_port=0
--tserver_master_addrs=127.11.253.254:35999
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.195.keytab
--principal=kudu/127.11.253.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.253.212:35221
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785988004637995-12279-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785988004637995-12279-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785988004637995-12279-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:50:56.547039 7389 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:50:56.547286 7389 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:50:56.547314 7389 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:50:56.548686 7389 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:50:56.548743 7389 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:50:56.548774 7389 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.253.195
I20260806 03:50:56.550513 7389 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.253.212:35221
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785988004637995-12279-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785988004637995-12279-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.195.keytab
--principal=kudu/127.11.253.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.253.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785988004637995-12279-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.253.195
--webserver_port=0
--tserver_master_addrs=127.11.253.254:35999
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7389
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.253.195
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785988004637995-12279-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:50:56.550803 7389 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:50:56.551098 7389 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:50:56.556783 7405 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:50:56.556783 7399 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:50:56.557013 7389 server_base.cc:1034] running on GCE node
W20260806 03:50:56.557111 7398 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:50:56.557405 7389 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:50:56.557725 7389 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:50:56.558902 7389 hybrid_clock.cc:648] HybridClock initialized: now 1785988256558882 us; error 38 us; skew 500 ppm
Aug 06 03:50:56 dist-test-slave-6ntk krb5kdc[6882](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988256, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.253.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:50:56.561812 7389 init.cc:377] Logged in from keytab as kudu/127.11.253.195@KRBTEST.COM (short username kudu)
I20260806 03:50:56.562374 7389 webserver.cc:460] Webserver started at http://127.11.253.195:44131/ using document root <none> and password file <none>
I20260806 03:50:56.562628 7389 fs_manager.cc:359] Metadata directory not provided
I20260806 03:50:56.562701 7389 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:50:56.562866 7389 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:50:56.563998 7389 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785988004637995-12279-0/minicluster-data/ts-2/data/instance:
uuid: "708e31cee85c483f845a81746684a489"
format_stamp: "Formatted at 2026-08-06 03:50:56 on dist-test-slave-6ntk"
server_key: "05296f82b4e12f719942ab3683632710"
server_key_iv: "daf94873ded66a707e2a94c6afbf641b"
server_key_version: "encryptionkey@0"
I20260806 03:50:56.564383 7389 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785988004637995-12279-0/minicluster-data/ts-2/wal/instance:
uuid: "708e31cee85c483f845a81746684a489"
format_stamp: "Formatted at 2026-08-06 03:50:56 on dist-test-slave-6ntk"
server_key: "05296f82b4e12f719942ab3683632710"
server_key_iv: "daf94873ded66a707e2a94c6afbf641b"
server_key_version: "encryptionkey@0"
I20260806 03:50:56.565975 7389 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260806 03:50:56.566869 7413 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:50:56.567113 7389 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 03:50:56.567236 7389 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785988004637995-12279-0/minicluster-data/ts-2/data,/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785988004637995-12279-0/minicluster-data/ts-2/wal
uuid: "708e31cee85c483f845a81746684a489"
format_stamp: "Formatted at 2026-08-06 03:50:56 on dist-test-slave-6ntk"
server_key: "05296f82b4e12f719942ab3683632710"
server_key_iv: "daf94873ded66a707e2a94c6afbf641b"
server_key_version: "encryptionkey@0"
I20260806 03:50:56.567310 7389 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785988004637995-12279-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785988004637995-12279-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785988004637995-12279-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:50:56.581602 7389 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:50:56.584219 7389 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:50:56.584403 7389 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:50:56.584642 7389 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 03:50:56.584968 7389 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 03:50:56.585009 7389 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 03:50:56.585054 7389 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 03:50:56.585081 7389 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 03:50:56.591331 7389 rpc_server.cc:307] RPC server started. Bound to: 127.11.253.195:44557
I20260806 03:50:56.591437 7562 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.253.195:44557 every 8 connection(s)
I20260806 03:50:56.591823 7389 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785988004637995-12279-0/minicluster-data/ts-2/data/info.pb
I20260806 03:50:56.593849 12279 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskToVWRe/build/release/bin/kudu as pid 7389
I20260806 03:50:56.593986 12279 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785988004637995-12279-0/minicluster-data/ts-2/wal/instance
I20260806 03:50:56.594163 12279 external_mini_cluster.cc:1428] Setting key 2f0345a89ecb055bb368811ca9490d3a
Aug 06 03:50:56 dist-test-slave-6ntk krb5kdc[6882](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988256, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.253.195@KRBTEST.COM for kudu/127.11.253.254@KRBTEST.COM
I20260806 03:50:56.602270 7012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:50:56.592918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.253.195:51849 (local address 127.11.253.254:35999)
0806 03:50:56.592988 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0806 03:50:56.592991 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 03:50:56.593230 (+ 239us) server_negotiation.cc:407] Connection header received
0806 03:50:56.593732 (+ 502us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:50:56.593734 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:50:56.593747 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:50:56.593832 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:50:56.595168 (+ 1336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:50:56.595774 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:50:56.596857 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:50:56.597052 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:50:56.597108 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:50:56.599370 (+ 2262us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:50:56.599371 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:50:56.599371 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:50:56.599378 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:50:56.600633 (+ 1255us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:50:56.601167 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:50:56.601168 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:50:56.601168 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:50:56.601226 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:50:56.601574 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:50:56.601575 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:50:56.601575 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:50:56.601727 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:50:56.601788 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0806 03:50:56.602132 (+ 344us) server_negotiation.cc:299] Negotiation successful
0806 03:50:56.602201 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260806 03:50:56.602460 7566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:50:56.592916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:35999 (local address 127.11.253.195:51849)
0806 03:50:56.593102 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0806 03:50:56.593141 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0806 03:50:56.593627 (+ 486us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:50:56.593985 (+ 358us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:50:56.593990 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:50:56.594843 (+ 853us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:50:56.595074 (+ 231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:50:56.595079 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:50:56.595877 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:50:56.595878 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:50:56.596781 (+ 903us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:50:56.596783 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:50:56.597126 (+ 343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:50:56.597127 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:50:56.597225 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:50:56.597776 (+ 551us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:50:56.597785 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:50:56.599278 (+ 1493us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:50:56.600724 (+ 1446us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:50:56.600730 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:50:56.600732 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:50:56.601037 (+ 305us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:50:56.601315 (+ 278us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:50:56.601315 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:50:56.601316 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:50:56.601486 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:50:56.601809 (+ 323us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:50:56.601820 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:50:56.602008 (+ 188us) client_negotiation.cc:769] Sending connection context
0806 03:50:56.602102 (+ 94us) client_negotiation.cc:240] Negotiation successful
0806 03:50:56.602230 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":92,"threads_started":1}
I20260806 03:50:56.602974 7563 heartbeater.cc:347] Connected to a master server at 127.11.253.254:35999
I20260806 03:50:56.603089 7563 heartbeater.cc:464] Registering TS with master...
I20260806 03:50:56.603277 7563 heartbeater.cc:511] Master 127.11.253.254:35999 requested a full tablet report, sending...
I20260806 03:50:56.603681 6944 ts_manager.cc:194] Registered new tserver with Master: 708e31cee85c483f845a81746684a489 (127.11.253.195:44557)
I20260806 03:50:56.604058 6944 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.253.195@KRBTEST.COM'} at 127.11.253.195:51849
I20260806 03:50:56.606122 12279 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 03:50:56.613269 7012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:50:56.607524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54824 (local address 127.11.253.254:35999)
0806 03:50:56.607571 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0806 03:50:56.607574 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 03:50:56.607649 (+ 75us) server_negotiation.cc:407] Connection header received
0806 03:50:56.607699 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:50:56.607700 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:50:56.607709 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:50:56.607755 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:50:56.608266 (+ 511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:50:56.608744 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:50:56.609480 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:50:56.609659 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:50:56.609714 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:50:56.611020 (+ 1306us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:50:56.611021 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:50:56.611021 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:50:56.611027 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:50:56.612248 (+ 1221us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:50:56.612597 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:50:56.612598 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:50:56.612598 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:50:56.612648 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:50:56.612812 (+ 164us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:50:56.612813 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:50:56.612813 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:50:56.612953 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:50:56.613031 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0806 03:50:56.613118 (+ 87us) server_negotiation.cc:299] Negotiation successful
0806 03:50:56.613176 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260806 03:50:56.614951 6944 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54824:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 03:50:56.616643 6944 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 03:50:56.623749 7577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:50:56.619524 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.193:41781 (local address 127.0.0.1:37000)
0806 03:50:56.619689 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0806 03:50:56.619701 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 03:50:56.619755 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:50:56.620188 (+ 433us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:50:56.620189 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:50:56.620191 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 03:50:56.620256 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:50:56.620258 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:50:56.621208 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:50:56.621210 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:50:56.622493 (+ 1283us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:50:56.622497 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:50:56.623392 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:50:56.623393 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:50:56.623482 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:50:56.623498 (+ 16us) client_negotiation.cc:769] Sending connection context
0806 03:50:56.623598 (+ 100us) client_negotiation.cc:240] Negotiation successful
0806 03:50:56.623664 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":76,"threads_started":1}
I20260806 03:50:56.623762 7582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:50:56.619766 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37000 (local address 127.11.253.193:41781)
0806 03:50:56.620005 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0806 03:50:56.620010 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 03:50:56.620021 (+ 11us) server_negotiation.cc:407] Connection header received
0806 03:50:56.620065 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:50:56.620067 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:50:56.620084 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:50:56.620368 (+ 284us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 03:50:56.620428 (+ 60us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:50:56.621088 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:50:56.622592 (+ 1504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:50:56.623285 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:50:56.623392 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:50:56.623469 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0806 03:50:56.623621 (+ 152us) server_negotiation.cc:299] Negotiation successful
0806 03:50:56.623665 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":199,"thread_start_us":106,"threads_started":1}
I20260806 03:50:56.623937 7581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:50:56.619347 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.194:43277 (local address 127.0.0.1:45738)
0806 03:50:56.619636 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0806 03:50:56.619646 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 03:50:56.619707 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:50:56.620129 (+ 422us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:50:56.620133 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:50:56.620135 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 03:50:56.620212 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:50:56.620215 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:50:56.621579 (+ 1364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:50:56.621580 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:50:56.622703 (+ 1123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:50:56.622706 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:50:56.623730 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:50:56.623730 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:50:56.623788 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:50:56.623791 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 03:50:56.623836 (+ 45us) client_negotiation.cc:240] Negotiation successful
0806 03:50:56.623868 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":270,"mutex_wait_us":11,"thread_start_us":106,"threads_started":1}
I20260806 03:50:56.624176 7580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:50:56.619496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45738 (local address 127.11.253.194:43277)
0806 03:50:56.619860 (+ 364us) server_negotiation.cc:206] Beginning negotiation
0806 03:50:56.619865 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 03:50:56.619876 (+ 11us) server_negotiation.cc:407] Connection header received
0806 03:50:56.619930 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:50:56.619932 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:50:56.619950 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:50:56.620028 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 03:50:56.620295 (+ 267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:50:56.621459 (+ 1164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:50:56.622820 (+ 1361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:50:56.623564 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:50:56.623645 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:50:56.623701 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0806 03:50:56.624048 (+ 347us) server_negotiation.cc:299] Negotiation successful
0806 03:50:56.624095 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":321,"thread_start_us":99,"threads_started":1}
I20260806 03:50:56.624234 7579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:50:56.619421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46470 (local address 127.11.253.195:44557)
0806 03:50:56.620083 (+ 662us) server_negotiation.cc:206] Beginning negotiation
0806 03:50:56.620087 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 03:50:56.620097 (+ 10us) server_negotiation.cc:407] Connection header received
0806 03:50:56.620133 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:50:56.620135 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:50:56.620150 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:50:56.620369 (+ 219us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 03:50:56.620459 (+ 90us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:50:56.621504 (+ 1045us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:50:56.622832 (+ 1328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:50:56.623565 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:50:56.623995 (+ 430us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:50:56.624071 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0806 03:50:56.624096 (+ 25us) server_negotiation.cc:299] Negotiation successful
0806 03:50:56.624144 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":622,"thread_start_us":79,"threads_started":1}
I20260806 03:50:56.623937 7578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:50:56.619298 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.195:44557 (local address 127.0.0.1:46470)
0806 03:50:56.619533 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0806 03:50:56.619577 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0806 03:50:56.619649 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:50:56.620249 (+ 600us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:50:56.620250 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:50:56.620252 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 03:50:56.620290 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:50:56.620291 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:50:56.621637 (+ 1346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:50:56.621638 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:50:56.622651 (+ 1013us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:50:56.622654 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:50:56.623675 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:50:56.623676 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:50:56.623743 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:50:56.623748 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 03:50:56.623780 (+ 32us) client_negotiation.cc:240] Negotiation successful
0806 03:50:56.623845 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":85,"threads_started":1}
I20260806 03:50:56.625222 7481 tablet_service.cc:1467] Processing CreateTablet for tablet fbb522d960be403487bb8dcab2b34a36 (DEFAULT_TABLE table=test-table [id=58a8266408a84a1482b546efd6d9d7f5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 03:50:56.625196 7300 tablet_service.cc:1467] Processing CreateTablet for tablet fbb522d960be403487bb8dcab2b34a36 (DEFAULT_TABLE table=test-table [id=58a8266408a84a1482b546efd6d9d7f5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 03:50:56.625363 7121 tablet_service.cc:1467] Processing CreateTablet for tablet fbb522d960be403487bb8dcab2b34a36 (DEFAULT_TABLE table=test-table [id=58a8266408a84a1482b546efd6d9d7f5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 03:50:56.625629 7300 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fbb522d960be403487bb8dcab2b34a36. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:50:56.625628 7481 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fbb522d960be403487bb8dcab2b34a36. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:50:56.625663 7121 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fbb522d960be403487bb8dcab2b34a36. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:50:56.628574 7583 tablet_bootstrap.cc:492] T fbb522d960be403487bb8dcab2b34a36 P 6a613dea1c854b1993d8ffe2f4529640: Bootstrap starting.
I20260806 03:50:56.629415 7583 tablet_bootstrap.cc:654] T fbb522d960be403487bb8dcab2b34a36 P 6a613dea1c854b1993d8ffe2f4529640: Neither blocks nor log segments found. Creating new log.
I20260806 03:50:56.629489 7586 tablet_bootstrap.cc:492] T fbb522d960be403487bb8dcab2b34a36 P ded1f9cb216243e4be42c68100817ef3: Bootstrap starting.
I20260806 03:50:56.630028 7585 tablet_bootstrap.cc:492] T fbb522d960be403487bb8dcab2b34a36 P 708e31cee85c483f845a81746684a489: Bootstrap starting.
I20260806 03:50:56.630095 7583 log.cc:824] T fbb522d960be403487bb8dcab2b34a36 P 6a613dea1c854b1993d8ffe2f4529640: Log is configured to *not* fsync() on all Append() calls
I20260806 03:50:56.630167 7586 tablet_bootstrap.cc:654] T fbb522d960be403487bb8dcab2b34a36 P ded1f9cb216243e4be42c68100817ef3: Neither blocks nor log segments found. Creating new log.
I20260806 03:50:56.630744 7585 tablet_bootstrap.cc:654] T fbb522d960be403487bb8dcab2b34a36 P 708e31cee85c483f845a81746684a489: Neither blocks nor log segments found. Creating new log.
I20260806 03:50:56.630748 7586 log.cc:824] T fbb522d960be403487bb8dcab2b34a36 P ded1f9cb216243e4be42c68100817ef3: Log is configured to *not* fsync() on all Append() calls
I20260806 03:50:56.630916 7583 tablet_bootstrap.cc:492] T fbb522d960be403487bb8dcab2b34a36 P 6a613dea1c854b1993d8ffe2f4529640: No bootstrap required, opened a new log
I20260806 03:50:56.631009 7583 ts_tablet_manager.cc:1397] T fbb522d960be403487bb8dcab2b34a36 P 6a613dea1c854b1993d8ffe2f4529640: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 03:50:56.631342 7585 log.cc:824] T fbb522d960be403487bb8dcab2b34a36 P 708e31cee85c483f845a81746684a489: Log is configured to *not* fsync() on all Append() calls
I20260806 03:50:56.631604 7586 tablet_bootstrap.cc:492] T fbb522d960be403487bb8dcab2b34a36 P ded1f9cb216243e4be42c68100817ef3: No bootstrap required, opened a new log
I20260806 03:50:56.631701 7586 ts_tablet_manager.cc:1397] T fbb522d960be403487bb8dcab2b34a36 P ded1f9cb216243e4be42c68100817ef3: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 03:50:56.632110 7585 tablet_bootstrap.cc:492] T fbb522d960be403487bb8dcab2b34a36 P 708e31cee85c483f845a81746684a489: No bootstrap required, opened a new log
I20260806 03:50:56.632320 7585 ts_tablet_manager.cc:1397] T fbb522d960be403487bb8dcab2b34a36 P 708e31cee85c483f845a81746684a489: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 03:50:56.633282 7583 raft_consensus.cc:348] T fbb522d960be403487bb8dcab2b34a36 P 6a613dea1c854b1993d8ffe2f4529640 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "708e31cee85c483f845a81746684a489" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 44557 } } peers { permanent_uuid: "ded1f9cb216243e4be42c68100817ef3" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 41781 } } peers { permanent_uuid: "6a613dea1c854b1993d8ffe2f4529640" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 43277 } }
I20260806 03:50:56.633476 7583 raft_consensus.cc:374] T fbb522d960be403487bb8dcab2b34a36 P 6a613dea1c854b1993d8ffe2f4529640 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:50:56.633517 7583 raft_consensus.cc:729] T fbb522d960be403487bb8dcab2b34a36 P 6a613dea1c854b1993d8ffe2f4529640 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a613dea1c854b1993d8ffe2f4529640, State: Initialized, Role: FOLLOWER
I20260806 03:50:56.633665 7583 consensus_queue.cc:260] T fbb522d960be403487bb8dcab2b34a36 P 6a613dea1c854b1993d8ffe2f4529640 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "708e31cee85c483f845a81746684a489" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 44557 } } peers { permanent_uuid: "ded1f9cb216243e4be42c68100817ef3" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 41781 } } peers { permanent_uuid: "6a613dea1c854b1993d8ffe2f4529640" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 43277 } }
I20260806 03:50:56.633641 7586 raft_consensus.cc:348] T fbb522d960be403487bb8dcab2b34a36 P ded1f9cb216243e4be42c68100817ef3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "708e31cee85c483f845a81746684a489" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 44557 } } peers { permanent_uuid: "ded1f9cb216243e4be42c68100817ef3" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 41781 } } peers { permanent_uuid: "6a613dea1c854b1993d8ffe2f4529640" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 43277 } }
I20260806 03:50:56.633828 7586 raft_consensus.cc:374] T fbb522d960be403487bb8dcab2b34a36 P ded1f9cb216243e4be42c68100817ef3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:50:56.633865 7586 raft_consensus.cc:729] T fbb522d960be403487bb8dcab2b34a36 P ded1f9cb216243e4be42c68100817ef3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ded1f9cb216243e4be42c68100817ef3, State: Initialized, Role: FOLLOWER
I20260806 03:50:56.633996 7583 ts_tablet_manager.cc:1428] T fbb522d960be403487bb8dcab2b34a36 P 6a613dea1c854b1993d8ffe2f4529640: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 03:50:56.634037 7586 consensus_queue.cc:260] T fbb522d960be403487bb8dcab2b34a36 P ded1f9cb216243e4be42c68100817ef3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "708e31cee85c483f845a81746684a489" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 44557 } } peers { permanent_uuid: "ded1f9cb216243e4be42c68100817ef3" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 41781 } } peers { permanent_uuid: "6a613dea1c854b1993d8ffe2f4529640" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 43277 } }
I20260806 03:50:56.634138 7381 heartbeater.cc:503] Master 127.11.253.254:35999 was elected leader, sending a full tablet report...
I20260806 03:50:56.634362 7586 ts_tablet_manager.cc:1428] T fbb522d960be403487bb8dcab2b34a36 P ded1f9cb216243e4be42c68100817ef3: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 03:50:56.634572 7208 heartbeater.cc:503] Master 127.11.253.254:35999 was elected leader, sending a full tablet report...
I20260806 03:50:56.634451 7585 raft_consensus.cc:348] T fbb522d960be403487bb8dcab2b34a36 P 708e31cee85c483f845a81746684a489 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "708e31cee85c483f845a81746684a489" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 44557 } } peers { permanent_uuid: "ded1f9cb216243e4be42c68100817ef3" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 41781 } } peers { permanent_uuid: "6a613dea1c854b1993d8ffe2f4529640" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 43277 } }
I20260806 03:50:56.634632 7585 raft_consensus.cc:374] T fbb522d960be403487bb8dcab2b34a36 P 708e31cee85c483f845a81746684a489 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:50:56.634685 7585 raft_consensus.cc:729] T fbb522d960be403487bb8dcab2b34a36 P 708e31cee85c483f845a81746684a489 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 708e31cee85c483f845a81746684a489, State: Initialized, Role: FOLLOWER
I20260806 03:50:56.634807 7585 consensus_queue.cc:260] T fbb522d960be403487bb8dcab2b34a36 P 708e31cee85c483f845a81746684a489 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "708e31cee85c483f845a81746684a489" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 44557 } } peers { permanent_uuid: "ded1f9cb216243e4be42c68100817ef3" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 41781 } } peers { permanent_uuid: "6a613dea1c854b1993d8ffe2f4529640" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 43277 } }
I20260806 03:50:56.635099 7585 ts_tablet_manager.cc:1428] T fbb522d960be403487bb8dcab2b34a36 P 708e31cee85c483f845a81746684a489: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 03:50:56.635248 7563 heartbeater.cc:503] Master 127.11.253.254:35999 was elected leader, sending a full tablet report...
I20260806 03:50:56.637032 7594 raft_consensus.cc:482] T fbb522d960be403487bb8dcab2b34a36 P 708e31cee85c483f845a81746684a489 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 03:50:56.637151 7594 raft_consensus.cc:504] T fbb522d960be403487bb8dcab2b34a36 P 708e31cee85c483f845a81746684a489 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "708e31cee85c483f845a81746684a489" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 44557 } } peers { permanent_uuid: "ded1f9cb216243e4be42c68100817ef3" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 41781 } } peers { permanent_uuid: "6a613dea1c854b1993d8ffe2f4529640" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 43277 } }
I20260806 03:50:56.637506 7594 leader_election.cc:288] T fbb522d960be403487bb8dcab2b34a36 P 708e31cee85c483f845a81746684a489 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ded1f9cb216243e4be42c68100817ef3 (127.11.253.193:41781), 6a613dea1c854b1993d8ffe2f4529640 (127.11.253.194:43277)
I20260806 03:50:56.640326 7566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:50:56.637708 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.194:43277 (local address 127.11.253.195:43471)
0806 03:50:56.637747 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0806 03:50:56.637759 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 03:50:56.637820 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:50:56.637989 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:50:56.637990 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:50:56.637992 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 03:50:56.638090 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:50:56.638092 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:50:56.638720 (+ 628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:50:56.638721 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:50:56.639476 (+ 755us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:50:56.639478 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:50:56.640055 (+ 577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:50:56.640056 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:50:56.640135 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:50:56.640138 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 03:50:56.640210 (+ 72us) client_negotiation.cc:240] Negotiation successful
0806 03:50:56.640253 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":21}
I20260806 03:50:56.640329 7580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:50:56.637775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.253.195:43471 (local address 127.11.253.194:43277)
0806 03:50:56.637825 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0806 03:50:56.637828 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 03:50:56.637837 (+ 9us) server_negotiation.cc:407] Connection header received
0806 03:50:56.637910 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:50:56.637912 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:50:56.637925 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:50:56.637981 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 03:50:56.638170 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:50:56.638630 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:50:56.639578 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:50:56.639968 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:50:56.640058 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:50:56.640068 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0806 03:50:56.640228 (+ 160us) server_negotiation.cc:299] Negotiation successful
0806 03:50:56.640263 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260806 03:50:56.640697 7322 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbb522d960be403487bb8dcab2b34a36" candidate_uuid: "708e31cee85c483f845a81746684a489" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a613dea1c854b1993d8ffe2f4529640" is_pre_election: true
I20260806 03:50:56.640799 7582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:50:56.637820 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.253.195:45753 (local address 127.11.253.193:41781)
0806 03:50:56.637860 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0806 03:50:56.637862 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:50:56.638143 (+ 281us) server_negotiation.cc:407] Connection header received
0806 03:50:56.638301 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:50:56.638302 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:50:56.638313 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:50:56.638371 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 03:50:56.638517 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:50:56.638990 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:50:56.639909 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:50:56.640292 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:50:56.640378 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:50:56.640387 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0806 03:50:56.640696 (+ 309us) server_negotiation.cc:299] Negotiation successful
0806 03:50:56.640740 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 03:50:56.640794 7595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:50:56.637708 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.193:41781 (local address 127.11.253.195:45753)
0806 03:50:56.638079 (+ 371us) negotiation.cc:107] Waiting for socket to connect
0806 03:50:56.638090 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 03:50:56.638148 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:50:56.638374 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:50:56.638374 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:50:56.638376 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 03:50:56.638441 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:50:56.638443 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:50:56.639082 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:50:56.639083 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:50:56.639819 (+ 736us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:50:56.639820 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:50:56.640519 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:50:56.640520 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:50:56.640596 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:50:56.640600 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 03:50:56.640676 (+ 76us) client_negotiation.cc:240] Negotiation successful
0806 03:50:56.640717 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":349,"mutex_wait_us":201,"thread_start_us":96,"threads_started":1}
I20260806 03:50:56.640852 7322 raft_consensus.cc:2454] T fbb522d960be403487bb8dcab2b34a36 P 6a613dea1c854b1993d8ffe2f4529640 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 708e31cee85c483f845a81746684a489 in term 0.
I20260806 03:50:56.641062 7148 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbb522d960be403487bb8dcab2b34a36" candidate_uuid: "708e31cee85c483f845a81746684a489" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ded1f9cb216243e4be42c68100817ef3" is_pre_election: true
I20260806 03:50:56.641175 7148 raft_consensus.cc:2454] T fbb522d960be403487bb8dcab2b34a36 P ded1f9cb216243e4be42c68100817ef3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 708e31cee85c483f845a81746684a489 in term 0.
I20260806 03:50:56.641144 7414 leader_election.cc:302] T fbb522d960be403487bb8dcab2b34a36 P 708e31cee85c483f845a81746684a489 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a613dea1c854b1993d8ffe2f4529640, 708e31cee85c483f845a81746684a489; no voters:
I20260806 03:50:56.641443 7594 raft_consensus.cc:2781] T fbb522d960be403487bb8dcab2b34a36 P 708e31cee85c483f845a81746684a489 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 03:50:56.641497 7594 raft_consensus.cc:482] T fbb522d960be403487bb8dcab2b34a36 P 708e31cee85c483f845a81746684a489 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 03:50:56.641669 7594 raft_consensus.cc:3037] T fbb522d960be403487bb8dcab2b34a36 P 708e31cee85c483f845a81746684a489 [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:50:56.642810 7594 raft_consensus.cc:504] T fbb522d960be403487bb8dcab2b34a36 P 708e31cee85c483f845a81746684a489 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "708e31cee85c483f845a81746684a489" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 44557 } } peers { permanent_uuid: "ded1f9cb216243e4be42c68100817ef3" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 41781 } } peers { permanent_uuid: "6a613dea1c854b1993d8ffe2f4529640" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 43277 } }
I20260806 03:50:56.643042 7594 leader_election.cc:288] T fbb522d960be403487bb8dcab2b34a36 P 708e31cee85c483f845a81746684a489 [CANDIDATE]: Term 1 election: Requested vote from peers ded1f9cb216243e4be42c68100817ef3 (127.11.253.193:41781), 6a613dea1c854b1993d8ffe2f4529640 (127.11.253.194:43277)
I20260806 03:50:56.643149 7322 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbb522d960be403487bb8dcab2b34a36" candidate_uuid: "708e31cee85c483f845a81746684a489" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a613dea1c854b1993d8ffe2f4529640"
I20260806 03:50:56.643147 7148 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbb522d960be403487bb8dcab2b34a36" candidate_uuid: "708e31cee85c483f845a81746684a489" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ded1f9cb216243e4be42c68100817ef3"
I20260806 03:50:56.643224 7322 raft_consensus.cc:3037] T fbb522d960be403487bb8dcab2b34a36 P 6a613dea1c854b1993d8ffe2f4529640 [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:50:56.643265 7148 raft_consensus.cc:3037] T fbb522d960be403487bb8dcab2b34a36 P ded1f9cb216243e4be42c68100817ef3 [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:50:56.643953 7322 raft_consensus.cc:2454] T fbb522d960be403487bb8dcab2b34a36 P 6a613dea1c854b1993d8ffe2f4529640 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 708e31cee85c483f845a81746684a489 in term 1.
I20260806 03:50:56.644001 7148 raft_consensus.cc:2454] T fbb522d960be403487bb8dcab2b34a36 P ded1f9cb216243e4be42c68100817ef3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 708e31cee85c483f845a81746684a489 in term 1.
I20260806 03:50:56.644168 7414 leader_election.cc:302] T fbb522d960be403487bb8dcab2b34a36 P 708e31cee85c483f845a81746684a489 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a613dea1c854b1993d8ffe2f4529640, 708e31cee85c483f845a81746684a489; no voters:
I20260806 03:50:56.644261 7594 raft_consensus.cc:2781] T fbb522d960be403487bb8dcab2b34a36 P 708e31cee85c483f845a81746684a489 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 03:50:56.644407 7594 raft_consensus.cc:686] T fbb522d960be403487bb8dcab2b34a36 P 708e31cee85c483f845a81746684a489 [term 1 LEADER]: Becoming Leader. State: Replica: 708e31cee85c483f845a81746684a489, State: Running, Role: LEADER
I20260806 03:50:56.644528 7594 consensus_queue.cc:237] T fbb522d960be403487bb8dcab2b34a36 P 708e31cee85c483f845a81746684a489 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "708e31cee85c483f845a81746684a489" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 44557 } } peers { permanent_uuid: "ded1f9cb216243e4be42c68100817ef3" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 41781 } } peers { permanent_uuid: "6a613dea1c854b1993d8ffe2f4529640" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 43277 } }
I20260806 03:50:56.645308 6944 catalog_manager.cc:5495] T fbb522d960be403487bb8dcab2b34a36 P 708e31cee85c483f845a81746684a489 reported cstate change: term changed from 0 to 1, leader changed from <none> to 708e31cee85c483f845a81746684a489 (127.11.253.195). New cstate: current_term: 1 leader_uuid: "708e31cee85c483f845a81746684a489" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "708e31cee85c483f845a81746684a489" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 44557 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ded1f9cb216243e4be42c68100817ef3" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 41781 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6a613dea1c854b1993d8ffe2f4529640" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 43277 } health_report { overall_health: UNKNOWN } } }
I20260806 03:50:56.652776 7579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:50:56.650468 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46478 (local address 127.11.253.195:44557)
0806 03:50:56.650534 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0806 03:50:56.650536 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:50:56.650545 (+ 9us) server_negotiation.cc:407] Connection header received
0806 03:50:56.650577 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:50:56.650578 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:50:56.650594 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:50:56.650659 (+ 65us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 03:50:56.650781 (+ 122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:50:56.651249 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:50:56.652114 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:50:56.652300 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:50:56.652356 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:50:56.652497 (+ 141us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 03:50:56.652595 (+ 98us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 03:50:56.652656 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0806 03:50:56.652688 (+ 32us) server_negotiation.cc:299] Negotiation successful
0806 03:50:56.652705 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260806 03:50:56.654775 7322 raft_consensus.cc:1261] T fbb522d960be403487bb8dcab2b34a36 P 6a613dea1c854b1993d8ffe2f4529640 [term 1 FOLLOWER]: Refusing update from remote peer 708e31cee85c483f845a81746684a489: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 03:50:56.654843 7148 raft_consensus.cc:1261] T fbb522d960be403487bb8dcab2b34a36 P ded1f9cb216243e4be42c68100817ef3 [term 1 FOLLOWER]: Refusing update from remote peer 708e31cee85c483f845a81746684a489: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 03:50:56.655189 7594 consensus_queue.cc:1035] T fbb522d960be403487bb8dcab2b34a36 P 708e31cee85c483f845a81746684a489 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a613dea1c854b1993d8ffe2f4529640" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 43277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 03:50:56.655370 7597 consensus_queue.cc:1035] T fbb522d960be403487bb8dcab2b34a36 P 708e31cee85c483f845a81746684a489 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ded1f9cb216243e4be42c68100817ef3" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 41781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 03:50:56.657719 7602 mvcc.cc:205] Tried to move back new op lower bound from 7315407899254996992 to 7315407899216662528. Current Snapshot: MvccSnapshot[applied={T|T < 7315407899254996992}]
I20260806 03:50:56.657975 7603 mvcc.cc:205] Tried to move back new op lower bound from 7315407899254996992 to 7315407899216662528. Current Snapshot: MvccSnapshot[applied={T|T < 7315407899254996992}]
I20260806 03:50:56.658318 7604 mvcc.cc:205] Tried to move back new op lower bound from 7315407899254996992 to 7315407899216662528. Current Snapshot: MvccSnapshot[applied={T|T < 7315407899254996992}]
I20260806 03:50:56.660715 6944 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54824:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 03:50:56.660835 6944 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54824:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 03:50:56.661549 6944 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a24df93ed8446a7ad7ea54a614d1ef3: Sending DeleteTablet for 3 replicas of tablet fbb522d960be403487bb8dcab2b34a36
I20260806 03:50:56.661857 7481 tablet_service.cc:1514] Processing DeleteTablet for tablet fbb522d960be403487bb8dcab2b34a36 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 03:50:56 UTC) from {username='kudu', principal='kudu/127.11.253.254@KRBTEST.COM'} at 127.0.0.1:46470
I20260806 03:50:56.661991 7300 tablet_service.cc:1514] Processing DeleteTablet for tablet fbb522d960be403487bb8dcab2b34a36 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 03:50:56 UTC) from {username='kudu', principal='kudu/127.11.253.254@KRBTEST.COM'} at 127.0.0.1:45738
I20260806 03:50:56.662009 7121 tablet_service.cc:1514] Processing DeleteTablet for tablet fbb522d960be403487bb8dcab2b34a36 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 03:50:56 UTC) from {username='kudu', principal='kudu/127.11.253.254@KRBTEST.COM'} at 127.0.0.1:37000
I20260806 03:50:56.662247 7618 tablet_replica.cc:330] T fbb522d960be403487bb8dcab2b34a36 P 6a613dea1c854b1993d8ffe2f4529640: stopping tablet replica
I20260806 03:50:56.662359 7618 raft_consensus.cc:2229] T fbb522d960be403487bb8dcab2b34a36 P 6a613dea1c854b1993d8ffe2f4529640 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 03:50:56.662469 7618 raft_consensus.cc:2258] T fbb522d960be403487bb8dcab2b34a36 P 6a613dea1c854b1993d8ffe2f4529640 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 03:50:56.662487 12279 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskToVWRe/build/release/bin/kudu with pid 7036
I20260806 03:50:56.669508 7615 tablet_replica.cc:330] T fbb522d960be403487bb8dcab2b34a36 P 708e31cee85c483f845a81746684a489: stopping tablet replica
I20260806 03:50:56.669637 7615 raft_consensus.cc:2229] T fbb522d960be403487bb8dcab2b34a36 P 708e31cee85c483f845a81746684a489 [term 1 LEADER]: Raft consensus shutting down.
I20260806 03:50:56.669783 7615 raft_consensus.cc:2258] T fbb522d960be403487bb8dcab2b34a36 P 708e31cee85c483f845a81746684a489 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 03:50:56.670034 7618 ts_tablet_manager.cc:1905] T fbb522d960be403487bb8dcab2b34a36 P 6a613dea1c854b1993d8ffe2f4529640: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 03:50:56.670429 7615 ts_tablet_manager.cc:1905] T fbb522d960be403487bb8dcab2b34a36 P 708e31cee85c483f845a81746684a489: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 03:50:56.672549 7615 ts_tablet_manager.cc:1918] T fbb522d960be403487bb8dcab2b34a36 P 708e31cee85c483f845a81746684a489: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 03:50:56.672629 7615 log.cc:1196] T fbb522d960be403487bb8dcab2b34a36 P 708e31cee85c483f845a81746684a489: Deleting WAL directory at /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785988004637995-12279-0/minicluster-data/ts-2/wal/wals/fbb522d960be403487bb8dcab2b34a36
I20260806 03:50:56.672567 7618 ts_tablet_manager.cc:1918] T fbb522d960be403487bb8dcab2b34a36 P 6a613dea1c854b1993d8ffe2f4529640: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 03:50:56.672829 7618 log.cc:1196] T fbb522d960be403487bb8dcab2b34a36 P 6a613dea1c854b1993d8ffe2f4529640: Deleting WAL directory at /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785988004637995-12279-0/minicluster-data/ts-1/wal/wals/fbb522d960be403487bb8dcab2b34a36
I20260806 03:50:56.672964 7615 ts_tablet_manager.cc:1939] T fbb522d960be403487bb8dcab2b34a36 P 708e31cee85c483f845a81746684a489: Deleting consensus metadata
I20260806 03:50:56.673223 7618 ts_tablet_manager.cc:1939] T fbb522d960be403487bb8dcab2b34a36 P 6a613dea1c854b1993d8ffe2f4529640: Deleting consensus metadata
I20260806 03:50:56.673377 6928 catalog_manager.cc:4841] TS 708e31cee85c483f845a81746684a489 (127.11.253.195:44557): tablet fbb522d960be403487bb8dcab2b34a36 (table test-table [id=58a8266408a84a1482b546efd6d9d7f5]) successfully deleted
I20260806 03:50:56.673748 6925 catalog_manager.cc:4841] TS 6a613dea1c854b1993d8ffe2f4529640 (127.11.253.194:43277): tablet fbb522d960be403487bb8dcab2b34a36 (table test-table [id=58a8266408a84a1482b546efd6d9d7f5]) successfully deleted
W20260806 03:50:56.674892 6928 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.253.193:41781 (error 108)
I20260806 03:50:56.675477 7577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:50:56.675328 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.193:41781 (local address 127.0.0.1:37004)
0806 03:50:56.675391 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0806 03:50:56.675435 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.253.193:41781: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260806 03:50:56.675575 12279 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskToVWRe/build/release/bin/kudu with pid 7214
W20260806 03:50:56.678683 6928 catalog_manager.cc:4584] TS ded1f9cb216243e4be42c68100817ef3 (127.11.253.193:41781): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet fbb522d960be403487bb8dcab2b34a36: Network error: Client connection negotiation failed: client connection to 127.11.253.193:41781: connect: Connection refused (error 111)
I20260806 03:50:56.683811 12279 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskToVWRe/build/release/bin/kudu with pid 7389
I20260806 03:50:56.692983 12279 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskToVWRe/build/release/bin/kudu with pid 6901
2026-08-06T03:50:56Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2109 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-06T03:50:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T03:50:56Z Disabled control of system clock
I20260806 03:50:56.729408 12279 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskToVWRe/build/release/bin/kudu
/tmp/dist-test-taskToVWRe/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785988004637995-12279-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785988004637995-12279-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785988004637995-12279-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785988004637995-12279-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.253.254:38071
--webserver_interface=127.11.253.254
--webserver_port=0
--builtin_ntp_servers=127.11.253.212:39807
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.253.254:38071
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 03:50:56.820417 7626 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:50:56.820662 7626 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:50:56.820693 7626 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:50:56.821914 7626 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 03:50:56.821961 7626 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:50:56.821980 7626 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:50:56.821995 7626 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 03:50:56.822009 7626 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 03:50:56.823426 7626 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.253.212:39807
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785988004637995-12279-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785988004637995-12279-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.253.254:38071
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.253.254:38071
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785988004637995-12279-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.253.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7626
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785988004637995-12279-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:50:56.823747 7626 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:50:56.824106 7626 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:50:56.829445 7632 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:50:56.829457 7643 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:50:56.829658 7626 server_base.cc:1034] running on GCE node
W20260806 03:50:56.829478 7636 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:50:56.830041 7626 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:50:56.830367 7626 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:50:56.831523 7626 hybrid_clock.cc:648] HybridClock initialized: now 1785988256831512 us; error 34 us; skew 500 ppm
I20260806 03:50:56.832885 7626 webserver.cc:460] Webserver started at http://127.11.253.254:37833/ using document root <none> and password file <none>
I20260806 03:50:56.833150 7626 fs_manager.cc:359] Metadata directory not provided
I20260806 03:50:56.833200 7626 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:50:56.833339 7626 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:50:56.834666 7626 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785988004637995-12279-0/minicluster-data/master-0/data/instance:
uuid: "736b3a5168094436a61cdb4f74ec0958"
format_stamp: "Formatted at 2026-08-06 03:50:56 on dist-test-slave-6ntk"
server_key: "e591ceafd4efbce689693d9dd3fae757"
server_key_iv: "1745cb47f52c32e8fa594a04f4305230"
server_key_version: "encryptionkey@0"
I20260806 03:50:56.835074 7626 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785988004637995-12279-0/minicluster-data/master-0/wal/instance:
uuid: "736b3a5168094436a61cdb4f74ec0958"
format_stamp: "Formatted at 2026-08-06 03:50:56 on dist-test-slave-6ntk"
server_key: "e591ceafd4efbce689693d9dd3fae757"
server_key_iv: "1745cb47f52c32e8fa594a04f4305230"
server_key_version: "encryptionkey@0"
I20260806 03:50:56.836472 7626 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260806 03:50:56.837296 7649 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:50:56.837504 7626 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 03:50:56.837621 7626 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785988004637995-12279-0/minicluster-data/master-0/data,/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785988004637995-12279-0/minicluster-data/master-0/wal
uuid: "736b3a5168094436a61cdb4f74ec0958"
format_stamp: "Formatted at 2026-08-06 03:50:56 on dist-test-slave-6ntk"
server_key: "e591ceafd4efbce689693d9dd3fae757"
server_key_iv: "1745cb47f52c32e8fa594a04f4305230"
server_key_version: "encryptionkey@0"
I20260806 03:50:56.837699 7626 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785988004637995-12279-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785988004637995-12279-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785988004637995-12279-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:50:56.849169 7626 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:50:56.849498 7626 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:50:56.849654 7626 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:50:56.853478 7626 rpc_server.cc:307] RPC server started. Bound to: 127.11.253.254:38071
I20260806 03:50:56.853549 7710 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.253.254:38071 every 8 connection(s)
I20260806 03:50:56.853936 7626 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785988004637995-12279-0/minicluster-data/master-0/data/info.pb
I20260806 03:50:56.854579 12279 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskToVWRe/build/release/bin/kudu as pid 7626
I20260806 03:50:56.854669 12279 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785988004637995-12279-0/minicluster-data/master-0/wal/instance
I20260806 03:50:56.854903 12279 external_mini_cluster.cc:1428] Setting key cfbbe485fec596cca34317b7f9d0cd7d
I20260806 03:50:56.855116 7711 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:50:56.858500 7711 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 736b3a5168094436a61cdb4f74ec0958: Bootstrap starting.
I20260806 03:50:56.859205 7711 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 736b3a5168094436a61cdb4f74ec0958: Neither blocks nor log segments found. Creating new log.
I20260806 03:50:56.859540 7711 log.cc:824] T 00000000000000000000000000000000 P 736b3a5168094436a61cdb4f74ec0958: Log is configured to *not* fsync() on all Append() calls
I20260806 03:50:56.860394 7711 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 736b3a5168094436a61cdb4f74ec0958: No bootstrap required, opened a new log
I20260806 03:50:56.860877 7715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:50:56.855505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40140 (local address 127.11.253.254:38071)
0806 03:50:56.855703 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0806 03:50:56.855710 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 03:50:56.855743 (+ 33us) server_negotiation.cc:407] Connection header received
0806 03:50:56.856288 (+ 545us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:50:56.856295 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:50:56.856360 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:50:56.856473 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:50:56.856717 (+ 244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:50:56.857573 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:50:56.858681 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:50:56.859296 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:50:56.859433 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:50:56.860195 (+ 762us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:50:56.860199 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:50:56.860199 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 03:50:56.860378 (+ 179us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:50:56.860414 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 03:50:56.860425 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:50:56.860507 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0806 03:50:56.860603 (+ 96us) server_negotiation.cc:299] Negotiation successful
0806 03:50:56.860703 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":89,"threads_started":1}
I20260806 03:50:56.862675 7711 raft_consensus.cc:348] T 00000000000000000000000000000000 P 736b3a5168094436a61cdb4f74ec0958 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "736b3a5168094436a61cdb4f74ec0958" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 38071 } }
I20260806 03:50:56.862836 7711 raft_consensus.cc:374] T 00000000000000000000000000000000 P 736b3a5168094436a61cdb4f74ec0958 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:50:56.862892 7711 raft_consensus.cc:729] T 00000000000000000000000000000000 P 736b3a5168094436a61cdb4f74ec0958 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 736b3a5168094436a61cdb4f74ec0958, State: Initialized, Role: FOLLOWER
I20260806 03:50:56.863014 7711 consensus_queue.cc:260] T 00000000000000000000000000000000 P 736b3a5168094436a61cdb4f74ec0958 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "736b3a5168094436a61cdb4f74ec0958" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 38071 } }
I20260806 03:50:56.863113 7711 raft_consensus.cc:388] T 00000000000000000000000000000000 P 736b3a5168094436a61cdb4f74ec0958 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 03:50:56.863157 7711 raft_consensus.cc:482] T 00000000000000000000000000000000 P 736b3a5168094436a61cdb4f74ec0958 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 03:50:56.863206 7711 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 736b3a5168094436a61cdb4f74ec0958 [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:50:56.863759 7711 raft_consensus.cc:504] T 00000000000000000000000000000000 P 736b3a5168094436a61cdb4f74ec0958 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "736b3a5168094436a61cdb4f74ec0958" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 38071 } }
I20260806 03:50:56.863885 7711 leader_election.cc:302] T 00000000000000000000000000000000 P 736b3a5168094436a61cdb4f74ec0958 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 736b3a5168094436a61cdb4f74ec0958; no voters:
I20260806 03:50:56.864080 7711 leader_election.cc:288] T 00000000000000000000000000000000 P 736b3a5168094436a61cdb4f74ec0958 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 03:50:56.864140 7718 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 736b3a5168094436a61cdb4f74ec0958 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 03:50:56.864374 7711 sys_catalog.cc:564] T 00000000000000000000000000000000 P 736b3a5168094436a61cdb4f74ec0958 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 03:50:56.864363 7718 raft_consensus.cc:686] T 00000000000000000000000000000000 P 736b3a5168094436a61cdb4f74ec0958 [term 1 LEADER]: Becoming Leader. State: Replica: 736b3a5168094436a61cdb4f74ec0958, State: Running, Role: LEADER
I20260806 03:50:56.864637 7718 consensus_queue.cc:237] T 00000000000000000000000000000000 P 736b3a5168094436a61cdb4f74ec0958 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "736b3a5168094436a61cdb4f74ec0958" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 38071 } }
I20260806 03:50:56.865056 7720 sys_catalog.cc:455] T 00000000000000000000000000000000 P 736b3a5168094436a61cdb4f74ec0958 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 736b3a5168094436a61cdb4f74ec0958. Latest consensus state: current_term: 1 leader_uuid: "736b3a5168094436a61cdb4f74ec0958" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "736b3a5168094436a61cdb4f74ec0958" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 38071 } } }
I20260806 03:50:56.865145 7720 sys_catalog.cc:458] T 00000000000000000000000000000000 P 736b3a5168094436a61cdb4f74ec0958 [sys.catalog]: This master's current role is: LEADER
I20260806 03:50:56.865329 7719 sys_catalog.cc:455] T 00000000000000000000000000000000 P 736b3a5168094436a61cdb4f74ec0958 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "736b3a5168094436a61cdb4f74ec0958" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "736b3a5168094436a61cdb4f74ec0958" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 38071 } } }
I20260806 03:50:56.865476 7719 sys_catalog.cc:458] T 00000000000000000000000000000000 P 736b3a5168094436a61cdb4f74ec0958 [sys.catalog]: This master's current role is: LEADER
I20260806 03:50:56.865386 7729 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 03:50:56.866068 7729 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 03:50:56.867760 7729 catalog_manager.cc:1347] Generated new cluster ID: f52401af00d9499fa258695d9fdd9af2
I20260806 03:50:56.867815 7729 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 03:50:56.876638 7729 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 03:50:56.877154 7729 catalog_manager.cc:1504] Loading token signing keys...
I20260806 03:50:56.886157 7729 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 736b3a5168094436a61cdb4f74ec0958: Generated new TSK 0
I20260806 03:50:56.886402 7729 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 03:50:56.891491 12279 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskToVWRe/build/release/bin/kudu
/tmp/dist-test-taskToVWRe/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785988004637995-12279-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785988004637995-12279-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785988004637995-12279-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785988004637995-12279-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.253.193:0
--local_ip_for_outbound_sockets=127.11.253.193
--webserver_interface=127.11.253.193
--webserver_port=0
--tserver_master_addrs=127.11.253.254:38071
--builtin_ntp_servers=127.11.253.212:39807
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 03:50:56.981734 7744 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:50:56.982010 7744 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:50:56.982038 7744 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:50:56.983184 7744 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:50:56.983227 7744 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:50:56.983253 7744 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.253.193
I20260806 03:50:56.984609 7744 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.253.212:39807
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785988004637995-12279-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785988004637995-12279-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.253.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785988004637995-12279-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.253.193
--webserver_port=0
--tserver_master_addrs=127.11.253.254:38071
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7744
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.253.193
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785988004637995-12279-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:50:56.984869 7744 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:50:56.985170 7744 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:50:56.991439 7751 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:50:56.991484 7754 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:50:56.991484 7752 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:50:56.991623 7744 server_base.cc:1034] running on GCE node
I20260806 03:50:56.991938 7744 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:50:56.992203 7744 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:50:56.993368 7744 hybrid_clock.cc:648] HybridClock initialized: now 1785988256993354 us; error 32 us; skew 500 ppm
I20260806 03:50:56.994876 7744 webserver.cc:460] Webserver started at http://127.11.253.193:34539/ using document root <none> and password file <none>
I20260806 03:50:56.995123 7744 fs_manager.cc:359] Metadata directory not provided
I20260806 03:50:56.995193 7744 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:50:56.995362 7744 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:50:56.996469 7744 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785988004637995-12279-0/minicluster-data/ts-0/data/instance:
uuid: "6188d9fee8af43399bb2ae8edec0e864"
format_stamp: "Formatted at 2026-08-06 03:50:56 on dist-test-slave-6ntk"
server_key: "f4833ffca281db6219ec40080e314fd2"
server_key_iv: "379ff4e64ad2268a5d52dff691335978"
server_key_version: "encryptionkey@0"
I20260806 03:50:56.996856 7744 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785988004637995-12279-0/minicluster-data/ts-0/wal/instance:
uuid: "6188d9fee8af43399bb2ae8edec0e864"
format_stamp: "Formatted at 2026-08-06 03:50:56 on dist-test-slave-6ntk"
server_key: "f4833ffca281db6219ec40080e314fd2"
server_key_iv: "379ff4e64ad2268a5d52dff691335978"
server_key_version: "encryptionkey@0"
I20260806 03:50:56.998328 7744 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 03:50:56.999173 7762 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:50:56.999377 7744 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 03:50:56.999469 7744 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785988004637995-12279-0/minicluster-data/ts-0/data,/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785988004637995-12279-0/minicluster-data/ts-0/wal
uuid: "6188d9fee8af43399bb2ae8edec0e864"
format_stamp: "Formatted at 2026-08-06 03:50:56 on dist-test-slave-6ntk"
server_key: "f4833ffca281db6219ec40080e314fd2"
server_key_iv: "379ff4e64ad2268a5d52dff691335978"
server_key_version: "encryptionkey@0"
I20260806 03:50:56.999544 7744 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785988004637995-12279-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785988004637995-12279-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785988004637995-12279-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:50:57.010932 7744 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:50:57.011264 7744 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:50:57.011422 7744 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:50:57.011664 7744 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 03:50:57.012007 7744 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 03:50:57.012042 7744 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 03:50:57.012065 7744 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 03:50:57.012110 7744 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 03:50:57.018457 7744 rpc_server.cc:307] RPC server started. Bound to: 127.11.253.193:45409
I20260806 03:50:57.018505 7908 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.253.193:45409 every 8 connection(s)
I20260806 03:50:57.018867 7744 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785988004637995-12279-0/minicluster-data/ts-0/data/info.pb
I20260806 03:50:57.024238 7715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:50:57.019620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.253.193:54687 (local address 127.11.253.254:38071)
0806 03:50:57.019714 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0806 03:50:57.019717 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 03:50:57.019900 (+ 183us) server_negotiation.cc:407] Connection header received
0806 03:50:57.020390 (+ 490us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:50:57.020391 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:50:57.020402 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:50:57.020453 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:50:57.021220 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:50:57.021718 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:50:57.022733 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:50:57.022945 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:50:57.023010 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:50:57.023884 (+ 874us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:50:57.023886 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:50:57.023886 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 03:50:57.023890 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:50:57.023906 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 03:50:57.023909 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:50:57.023962 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0806 03:50:57.024130 (+ 168us) server_negotiation.cc:299] Negotiation successful
0806 03:50:57.024172 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260806 03:50:57.024499 7911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:50:57.019570 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:38071 (local address 127.11.253.193:54687)
0806 03:50:57.019773 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0806 03:50:57.019811 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0806 03:50:57.020306 (+ 495us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:50:57.020517 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:50:57.020520 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:50:57.020528 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:50:57.021136 (+ 608us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:50:57.021143 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:50:57.021813 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:50:57.021814 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:50:57.022623 (+ 809us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:50:57.022626 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:50:57.023039 (+ 413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:50:57.023040 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:50:57.023161 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:50:57.023723 (+ 562us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 03:50:57.023724 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:50:57.023755 (+ 31us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 03:50:57.023779 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:50:57.024009 (+ 230us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:50:57.024026 (+ 17us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:50:57.024029 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 03:50:57.024113 (+ 84us) client_negotiation.cc:240] Negotiation successful
0806 03:50:57.024206 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":104,"threads_started":1}
I20260806 03:50:57.025156 7909 heartbeater.cc:347] Connected to a master server at 127.11.253.254:38071
I20260806 03:50:57.025262 7909 heartbeater.cc:464] Registering TS with master...
I20260806 03:50:57.025475 7909 heartbeater.cc:511] Master 127.11.253.254:38071 requested a full tablet report, sending...
I20260806 03:50:57.026162 7669 ts_manager.cc:194] Registered new tserver with Master: 6188d9fee8af43399bb2ae8edec0e864 (127.11.253.193:45409)
I20260806 03:50:57.026808 7669 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.11.253.193:54687
I20260806 03:50:57.028493 12279 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskToVWRe/build/release/bin/kudu as pid 7744
I20260806 03:50:57.028617 12279 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785988004637995-12279-0/minicluster-data/ts-0/wal/instance
I20260806 03:50:57.028832 12279 external_mini_cluster.cc:1428] Setting key dea915d688abf14833c66a22241b65f8
I20260806 03:50:57.030022 12279 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskToVWRe/build/release/bin/kudu
/tmp/dist-test-taskToVWRe/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785988004637995-12279-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785988004637995-12279-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785988004637995-12279-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785988004637995-12279-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.253.194:0
--local_ip_for_outbound_sockets=127.11.253.194
--webserver_interface=127.11.253.194
--webserver_port=0
--tserver_master_addrs=127.11.253.254:38071
--builtin_ntp_servers=127.11.253.212:39807
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 03:50:57.117520 7915 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:50:57.117738 7915 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:50:57.117760 7915 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:50:57.118924 7915 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:50:57.118965 7915 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:50:57.118990 7915 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.253.194
I20260806 03:50:57.120332 7915 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.253.212:39807
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785988004637995-12279-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785988004637995-12279-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.253.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785988004637995-12279-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.253.194
--webserver_port=0
--tserver_master_addrs=127.11.253.254:38071
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7915
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.253.194
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785988004637995-12279-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:50:57.120585 7915 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:50:57.120895 7915 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:50:57.126207 7924 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:50:57.126278 7929 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:50:57.126209 7923 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:50:57.126488 7915 server_base.cc:1034] running on GCE node
I20260806 03:50:57.126776 7915 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:50:57.127050 7915 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:50:57.128226 7915 hybrid_clock.cc:648] HybridClock initialized: now 1785988257128175 us; error 68 us; skew 500 ppm
I20260806 03:50:57.129540 7915 webserver.cc:460] Webserver started at http://127.11.253.194:35503/ using document root <none> and password file <none>
I20260806 03:50:57.129794 7915 fs_manager.cc:359] Metadata directory not provided
I20260806 03:50:57.129885 7915 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:50:57.130062 7915 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:50:57.131132 7915 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785988004637995-12279-0/minicluster-data/ts-1/data/instance:
uuid: "e7cb445574c842acb6024125a3c1a213"
format_stamp: "Formatted at 2026-08-06 03:50:57 on dist-test-slave-6ntk"
server_key: "710bdcfa25d6c412e0fdf890008cc215"
server_key_iv: "affa109eab7b5f40e7d78b562114ffdd"
server_key_version: "encryptionkey@0"
I20260806 03:50:57.131506 7915 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785988004637995-12279-0/minicluster-data/ts-1/wal/instance:
uuid: "e7cb445574c842acb6024125a3c1a213"
format_stamp: "Formatted at 2026-08-06 03:50:57 on dist-test-slave-6ntk"
server_key: "710bdcfa25d6c412e0fdf890008cc215"
server_key_iv: "affa109eab7b5f40e7d78b562114ffdd"
server_key_version: "encryptionkey@0"
I20260806 03:50:57.132884 7915 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 03:50:57.133656 7936 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:50:57.133857 7915 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 03:50:57.134003 7915 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785988004637995-12279-0/minicluster-data/ts-1/data,/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785988004637995-12279-0/minicluster-data/ts-1/wal
uuid: "e7cb445574c842acb6024125a3c1a213"
format_stamp: "Formatted at 2026-08-06 03:50:57 on dist-test-slave-6ntk"
server_key: "710bdcfa25d6c412e0fdf890008cc215"
server_key_iv: "affa109eab7b5f40e7d78b562114ffdd"
server_key_version: "encryptionkey@0"
I20260806 03:50:57.134081 7915 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785988004637995-12279-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785988004637995-12279-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785988004637995-12279-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:50:57.144698 7915 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:50:57.144990 7915 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:50:57.145135 7915 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:50:57.145349 7915 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 03:50:57.145680 7915 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 03:50:57.145740 7915 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 03:50:57.145783 7915 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 03:50:57.145812 7915 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 03:50:57.151731 7915 rpc_server.cc:307] RPC server started. Bound to: 127.11.253.194:33263
I20260806 03:50:57.151782 8088 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.253.194:33263 every 8 connection(s)
I20260806 03:50:57.152096 7915 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785988004637995-12279-0/minicluster-data/ts-1/data/info.pb
I20260806 03:50:57.156541 12279 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskToVWRe/build/release/bin/kudu as pid 7915
I20260806 03:50:57.156675 12279 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785988004637995-12279-0/minicluster-data/ts-1/wal/instance
I20260806 03:50:57.156889 12279 external_mini_cluster.cc:1428] Setting key 5b21f6d00ffcee38cad7d2ba2aa6e83f
I20260806 03:50:57.157552 7715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:50:57.152807 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.253.194:32811 (local address 127.11.253.254:38071)
0806 03:50:57.152856 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0806 03:50:57.152859 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 03:50:57.153066 (+ 207us) server_negotiation.cc:407] Connection header received
0806 03:50:57.153634 (+ 568us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:50:57.153635 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:50:57.153649 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:50:57.153704 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:50:57.154481 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:50:57.155010 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:50:57.156017 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:50:57.156218 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:50:57.156299 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:50:57.157252 (+ 953us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:50:57.157254 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:50:57.157254 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 03:50:57.157258 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:50:57.157291 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 03:50:57.157297 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:50:57.157364 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0806 03:50:57.157440 (+ 76us) server_negotiation.cc:299] Negotiation successful
0806 03:50:57.157463 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260806 03:50:57.157785 8092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:50:57.152762 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:38071 (local address 127.11.253.194:32811)
0806 03:50:57.152945 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0806 03:50:57.152979 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0806 03:50:57.153521 (+ 542us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:50:57.153774 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:50:57.153779 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:50:57.153791 (+ 12us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:50:57.154374 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:50:57.154379 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:50:57.155118 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:50:57.155119 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:50:57.155913 (+ 794us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:50:57.155916 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:50:57.156315 (+ 399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:50:57.156317 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:50:57.156480 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:50:57.157066 (+ 586us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 03:50:57.157067 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:50:57.157109 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 03:50:57.157139 (+ 30us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:50:57.157372 (+ 233us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:50:57.157378 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:50:57.157380 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 03:50:57.157432 (+ 52us) client_negotiation.cc:240] Negotiation successful
0806 03:50:57.157537 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":75,"threads_started":1}
I20260806 03:50:57.158102 12279 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskToVWRe/build/release/bin/kudu
/tmp/dist-test-taskToVWRe/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785988004637995-12279-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785988004637995-12279-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785988004637995-12279-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785988004637995-12279-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.253.195:0
--local_ip_for_outbound_sockets=127.11.253.195
--webserver_interface=127.11.253.195
--webserver_port=0
--tserver_master_addrs=127.11.253.254:38071
--builtin_ntp_servers=127.11.253.212:39807
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260806 03:50:57.158250 8089 heartbeater.cc:347] Connected to a master server at 127.11.253.254:38071
I20260806 03:50:57.158366 8089 heartbeater.cc:464] Registering TS with master...
I20260806 03:50:57.158546 8089 heartbeater.cc:511] Master 127.11.253.254:38071 requested a full tablet report, sending...
I20260806 03:50:57.158900 7669 ts_manager.cc:194] Registered new tserver with Master: e7cb445574c842acb6024125a3c1a213 (127.11.253.194:33263)
I20260806 03:50:57.159281 7669 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.11.253.194:32811
W20260806 03:50:57.243158 8094 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:50:57.243395 8094 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:50:57.243431 8094 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:50:57.244560 8094 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:50:57.244623 8094 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:50:57.244661 8094 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.253.195
I20260806 03:50:57.246058 8094 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.253.212:39807
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785988004637995-12279-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785988004637995-12279-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.253.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785988004637995-12279-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.253.195
--webserver_port=0
--tserver_master_addrs=127.11.253.254:38071
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8094
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.253.195
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785988004637995-12279-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:50:57.246330 8094 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:50:57.246635 8094 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:50:57.252002 8100 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:50:57.252009 8103 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:50:57.252108 8094 server_base.cc:1034] running on GCE node
W20260806 03:50:57.252013 8105 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:50:57.252530 8094 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:50:57.252825 8094 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:50:57.253973 8094 hybrid_clock.cc:648] HybridClock initialized: now 1785988257253962 us; error 37 us; skew 500 ppm
I20260806 03:50:57.255440 8094 webserver.cc:460] Webserver started at http://127.11.253.195:40733/ using document root <none> and password file <none>
I20260806 03:50:57.255693 8094 fs_manager.cc:359] Metadata directory not provided
I20260806 03:50:57.255744 8094 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:50:57.255889 8094 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:50:57.257018 8094 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785988004637995-12279-0/minicluster-data/ts-2/data/instance:
uuid: "e016b269efd04889b423b8a8a15d3060"
format_stamp: "Formatted at 2026-08-06 03:50:57 on dist-test-slave-6ntk"
server_key: "4d2366b1c6eea3e33b8ae2c69d6fc807"
server_key_iv: "f119155a6af14f0688603e9b16d945f6"
server_key_version: "encryptionkey@0"
I20260806 03:50:57.257388 8094 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785988004637995-12279-0/minicluster-data/ts-2/wal/instance:
uuid: "e016b269efd04889b423b8a8a15d3060"
format_stamp: "Formatted at 2026-08-06 03:50:57 on dist-test-slave-6ntk"
server_key: "4d2366b1c6eea3e33b8ae2c69d6fc807"
server_key_iv: "f119155a6af14f0688603e9b16d945f6"
server_key_version: "encryptionkey@0"
I20260806 03:50:57.258885 8094 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 03:50:57.259760 8112 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:50:57.259963 8094 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 03:50:57.260075 8094 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785988004637995-12279-0/minicluster-data/ts-2/data,/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785988004637995-12279-0/minicluster-data/ts-2/wal
uuid: "e016b269efd04889b423b8a8a15d3060"
format_stamp: "Formatted at 2026-08-06 03:50:57 on dist-test-slave-6ntk"
server_key: "4d2366b1c6eea3e33b8ae2c69d6fc807"
server_key_iv: "f119155a6af14f0688603e9b16d945f6"
server_key_version: "encryptionkey@0"
I20260806 03:50:57.260154 8094 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785988004637995-12279-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785988004637995-12279-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785988004637995-12279-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:50:57.273403 8094 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:50:57.273705 8094 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:50:57.273918 8094 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:50:57.274121 8094 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 03:50:57.274387 8094 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 03:50:57.274417 8094 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 03:50:57.274439 8094 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 03:50:57.274452 8094 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 03:50:57.280349 8094 rpc_server.cc:307] RPC server started. Bound to: 127.11.253.195:46167
I20260806 03:50:57.280462 8272 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.253.195:46167 every 8 connection(s)
I20260806 03:50:57.280771 8094 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785988004637995-12279-0/minicluster-data/ts-2/data/info.pb
I20260806 03:50:57.284066 12279 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskToVWRe/build/release/bin/kudu as pid 8094
I20260806 03:50:57.284183 12279 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785988004637995-12279-0/minicluster-data/ts-2/wal/instance
I20260806 03:50:57.284389 12279 external_mini_cluster.cc:1428] Setting key 67094c9becc489c911a0c8ecb745e22d
I20260806 03:50:57.286657 7715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:50:57.281584 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.253.195:47779 (local address 127.11.253.254:38071)
0806 03:50:57.281657 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0806 03:50:57.281660 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 03:50:57.281914 (+ 254us) server_negotiation.cc:407] Connection header received
0806 03:50:57.282501 (+ 587us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:50:57.282502 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:50:57.282514 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:50:57.282578 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:50:57.283207 (+ 629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:50:57.283688 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:50:57.285081 (+ 1393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:50:57.285292 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:50:57.285383 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:50:57.286335 (+ 952us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:50:57.286336 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:50:57.286336 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 03:50:57.286341 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:50:57.286357 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 03:50:57.286362 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:50:57.286468 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0806 03:50:57.286550 (+ 82us) server_negotiation.cc:299] Negotiation successful
0806 03:50:57.286577 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260806 03:50:57.286864 8277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:50:57.281573 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:38071 (local address 127.11.253.195:47779)
0806 03:50:57.281771 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0806 03:50:57.281807 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0806 03:50:57.282398 (+ 591us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:50:57.282630 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:50:57.282634 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:50:57.282642 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:50:57.283138 (+ 496us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:50:57.283141 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:50:57.283770 (+ 629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:50:57.283771 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:50:57.284992 (+ 1221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:50:57.284994 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:50:57.285394 (+ 400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:50:57.285395 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:50:57.285540 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:50:57.286167 (+ 627us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 03:50:57.286168 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:50:57.286196 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 03:50:57.286220 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:50:57.286468 (+ 248us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:50:57.286476 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:50:57.286479 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 03:50:57.286543 (+ 64us) client_negotiation.cc:240] Negotiation successful
0806 03:50:57.286642 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":110,"threads_started":1}
I20260806 03:50:57.287319 8274 heartbeater.cc:347] Connected to a master server at 127.11.253.254:38071
I20260806 03:50:57.287413 8274 heartbeater.cc:464] Registering TS with master...
I20260806 03:50:57.287621 8274 heartbeater.cc:511] Master 127.11.253.254:38071 requested a full tablet report, sending...
I20260806 03:50:57.287993 7669 ts_manager.cc:194] Registered new tserver with Master: e016b269efd04889b423b8a8a15d3060 (127.11.253.195:46167)
I20260806 03:50:57.288334 7669 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.11.253.195:47779
I20260806 03:50:57.296216 12279 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 03:50:57.300657 7715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:50:57.297679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40146 (local address 127.11.253.254:38071)
0806 03:50:57.297773 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0806 03:50:57.297775 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:50:57.297856 (+ 81us) server_negotiation.cc:407] Connection header received
0806 03:50:57.297952 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:50:57.297952 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:50:57.297963 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:50:57.298010 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:50:57.298175 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:50:57.298661 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:50:57.299475 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:50:57.299691 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:50:57.299780 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:50:57.300424 (+ 644us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:50:57.300426 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:50:57.300426 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 03:50:57.300431 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:50:57.300446 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 03:50:57.300450 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:50:57.300498 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0806 03:50:57.300548 (+ 50us) server_negotiation.cc:299] Negotiation successful
0806 03:50:57.300571 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260806 03:50:57.302380 7669 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:40146:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 03:50:57.304108 7669 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 03:50:57.310675 8287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:50:57.306554 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.193:45409 (local address 127.0.0.1:33392)
0806 03:50:57.306754 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0806 03:50:57.306815 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0806 03:50:57.306886 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:50:57.307239 (+ 353us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:50:57.307243 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:50:57.307245 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 03:50:57.307312 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:50:57.307314 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:50:57.308255 (+ 941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:50:57.308256 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:50:57.309370 (+ 1114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:50:57.309373 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:50:57.310349 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:50:57.310350 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:50:57.310444 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:50:57.310460 (+ 16us) client_negotiation.cc:769] Sending connection context
0806 03:50:57.310529 (+ 69us) client_negotiation.cc:240] Negotiation successful
0806 03:50:57.310582 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":79,"threads_started":1}
I20260806 03:50:57.310830 8289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:50:57.306645 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59924 (local address 127.11.253.194:33263)
0806 03:50:57.307111 (+ 466us) server_negotiation.cc:206] Beginning negotiation
0806 03:50:57.307115 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 03:50:57.307127 (+ 12us) server_negotiation.cc:407] Connection header received
0806 03:50:57.307163 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:50:57.307165 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:50:57.307180 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:50:57.307270 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 03:50:57.307483 (+ 213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:50:57.308676 (+ 1193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:50:57.309782 (+ 1106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:50:57.310457 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:50:57.310540 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:50:57.310624 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0806 03:50:57.310686 (+ 62us) server_negotiation.cc:299] Negotiation successful
0806 03:50:57.310733 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":423,"thread_start_us":87,"threads_started":1}
I20260806 03:50:57.311170 8292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:50:57.306839 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33392 (local address 127.11.253.193:45409)
0806 03:50:57.307023 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0806 03:50:57.307027 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 03:50:57.307038 (+ 11us) server_negotiation.cc:407] Connection header received
0806 03:50:57.307081 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:50:57.307083 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:50:57.307099 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:50:57.307190 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 03:50:57.307408 (+ 218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:50:57.308151 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:50:57.309439 (+ 1288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:50:57.310221 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:50:57.310323 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:50:57.310416 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0806 03:50:57.311027 (+ 611us) server_negotiation.cc:299] Negotiation successful
0806 03:50:57.311063 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153,"thread_start_us":82,"threads_started":1}
I20260806 03:50:57.311286 8290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:50:57.306827 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38114 (local address 127.11.253.195:46167)
0806 03:50:57.307015 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0806 03:50:57.307020 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 03:50:57.307411 (+ 391us) server_negotiation.cc:407] Connection header received
0806 03:50:57.307454 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:50:57.307457 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:50:57.307473 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:50:57.307544 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 03:50:57.307734 (+ 190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:50:57.308788 (+ 1054us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:50:57.310017 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:50:57.310805 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:50:57.310875 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:50:57.310960 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0806 03:50:57.311143 (+ 183us) server_negotiation.cc:299] Negotiation successful
0806 03:50:57.311189 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":151,"thread_start_us":81,"threads_started":1}
I20260806 03:50:57.310804 8288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:50:57.306555 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.194:33263 (local address 127.0.0.1:59924)
0806 03:50:57.306825 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0806 03:50:57.306836 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 03:50:57.306886 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:50:57.307347 (+ 461us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:50:57.307348 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:50:57.307349 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 03:50:57.307401 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:50:57.307402 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:50:57.308769 (+ 1367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:50:57.308770 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:50:57.309696 (+ 926us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:50:57.309698 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:50:57.310550 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:50:57.310551 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:50:57.310615 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:50:57.310620 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 03:50:57.310675 (+ 55us) client_negotiation.cc:240] Negotiation successful
0806 03:50:57.310723 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":255,"mutex_wait_us":2,"thread_start_us":95,"threads_started":1}
I20260806 03:50:57.311167 8284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:50:57.306580 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.195:46167 (local address 127.0.0.1:38114)
0806 03:50:57.307288 (+ 708us) negotiation.cc:107] Waiting for socket to connect
0806 03:50:57.307299 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 03:50:57.307349 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:50:57.307608 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:50:57.307609 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:50:57.307611 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 03:50:57.307642 (+ 31us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:50:57.307644 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:50:57.308850 (+ 1206us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:50:57.308851 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:50:57.309919 (+ 1068us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:50:57.309921 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:50:57.310994 (+ 1073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:50:57.310994 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:50:57.311047 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:50:57.311051 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 03:50:57.311095 (+ 44us) client_negotiation.cc:240] Negotiation successful
0806 03:50:57.311124 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":694,"thread_start_us":87,"threads_started":1}
I20260806 03:50:57.312052 7827 tablet_service.cc:1467] Processing CreateTablet for tablet eee0ea2ed3b441468de9f3b7b018ffdf (DEFAULT_TABLE table=test-table [id=12faa6467b0f467caa1514301e5beac0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 03:50:57.312050 7997 tablet_service.cc:1467] Processing CreateTablet for tablet eee0ea2ed3b441468de9f3b7b018ffdf (DEFAULT_TABLE table=test-table [id=12faa6467b0f467caa1514301e5beac0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 03:50:57.312440 7997 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eee0ea2ed3b441468de9f3b7b018ffdf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:50:57.312440 7827 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eee0ea2ed3b441468de9f3b7b018ffdf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:50:57.312345 8184 tablet_service.cc:1467] Processing CreateTablet for tablet eee0ea2ed3b441468de9f3b7b018ffdf (DEFAULT_TABLE table=test-table [id=12faa6467b0f467caa1514301e5beac0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 03:50:57.312690 8184 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eee0ea2ed3b441468de9f3b7b018ffdf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:50:57.315558 8293 tablet_bootstrap.cc:492] T eee0ea2ed3b441468de9f3b7b018ffdf P e016b269efd04889b423b8a8a15d3060: Bootstrap starting.
I20260806 03:50:57.315860 8297 tablet_bootstrap.cc:492] T eee0ea2ed3b441468de9f3b7b018ffdf P 6188d9fee8af43399bb2ae8edec0e864: Bootstrap starting.
I20260806 03:50:57.316085 8295 tablet_bootstrap.cc:492] T eee0ea2ed3b441468de9f3b7b018ffdf P e7cb445574c842acb6024125a3c1a213: Bootstrap starting.
I20260806 03:50:57.316283 8293 tablet_bootstrap.cc:654] T eee0ea2ed3b441468de9f3b7b018ffdf P e016b269efd04889b423b8a8a15d3060: Neither blocks nor log segments found. Creating new log.
I20260806 03:50:57.316625 8293 log.cc:824] T eee0ea2ed3b441468de9f3b7b018ffdf P e016b269efd04889b423b8a8a15d3060: Log is configured to *not* fsync() on all Append() calls
I20260806 03:50:57.316772 8295 tablet_bootstrap.cc:654] T eee0ea2ed3b441468de9f3b7b018ffdf P e7cb445574c842acb6024125a3c1a213: Neither blocks nor log segments found. Creating new log.
I20260806 03:50:57.317058 8295 log.cc:824] T eee0ea2ed3b441468de9f3b7b018ffdf P e7cb445574c842acb6024125a3c1a213: Log is configured to *not* fsync() on all Append() calls
I20260806 03:50:57.317682 8297 tablet_bootstrap.cc:654] T eee0ea2ed3b441468de9f3b7b018ffdf P 6188d9fee8af43399bb2ae8edec0e864: Neither blocks nor log segments found. Creating new log.
I20260806 03:50:57.317756 8295 tablet_bootstrap.cc:492] T eee0ea2ed3b441468de9f3b7b018ffdf P e7cb445574c842acb6024125a3c1a213: No bootstrap required, opened a new log
I20260806 03:50:57.317916 8295 ts_tablet_manager.cc:1397] T eee0ea2ed3b441468de9f3b7b018ffdf P e7cb445574c842acb6024125a3c1a213: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 03:50:57.318074 8297 log.cc:824] T eee0ea2ed3b441468de9f3b7b018ffdf P 6188d9fee8af43399bb2ae8edec0e864: Log is configured to *not* fsync() on all Append() calls
I20260806 03:50:57.318441 8293 tablet_bootstrap.cc:492] T eee0ea2ed3b441468de9f3b7b018ffdf P e016b269efd04889b423b8a8a15d3060: No bootstrap required, opened a new log
I20260806 03:50:57.318529 8293 ts_tablet_manager.cc:1397] T eee0ea2ed3b441468de9f3b7b018ffdf P e016b269efd04889b423b8a8a15d3060: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260806 03:50:57.318903 8297 tablet_bootstrap.cc:492] T eee0ea2ed3b441468de9f3b7b018ffdf P 6188d9fee8af43399bb2ae8edec0e864: No bootstrap required, opened a new log
I20260806 03:50:57.318975 8297 ts_tablet_manager.cc:1397] T eee0ea2ed3b441468de9f3b7b018ffdf P 6188d9fee8af43399bb2ae8edec0e864: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20260806 03:50:57.320180 8295 raft_consensus.cc:348] T eee0ea2ed3b441468de9f3b7b018ffdf P e7cb445574c842acb6024125a3c1a213 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6188d9fee8af43399bb2ae8edec0e864" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 45409 } } peers { permanent_uuid: "e7cb445574c842acb6024125a3c1a213" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 33263 } } peers { permanent_uuid: "e016b269efd04889b423b8a8a15d3060" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 46167 } }
I20260806 03:50:57.320359 8295 raft_consensus.cc:374] T eee0ea2ed3b441468de9f3b7b018ffdf P e7cb445574c842acb6024125a3c1a213 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:50:57.320400 8295 raft_consensus.cc:729] T eee0ea2ed3b441468de9f3b7b018ffdf P e7cb445574c842acb6024125a3c1a213 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7cb445574c842acb6024125a3c1a213, State: Initialized, Role: FOLLOWER
I20260806 03:50:57.320513 8295 consensus_queue.cc:260] T eee0ea2ed3b441468de9f3b7b018ffdf P e7cb445574c842acb6024125a3c1a213 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6188d9fee8af43399bb2ae8edec0e864" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 45409 } } peers { permanent_uuid: "e7cb445574c842acb6024125a3c1a213" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 33263 } } peers { permanent_uuid: "e016b269efd04889b423b8a8a15d3060" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 46167 } }
I20260806 03:50:57.320819 8295 ts_tablet_manager.cc:1428] T eee0ea2ed3b441468de9f3b7b018ffdf P e7cb445574c842acb6024125a3c1a213: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 03:50:57.320981 8089 heartbeater.cc:503] Master 127.11.253.254:38071 was elected leader, sending a full tablet report...
I20260806 03:50:57.321604 8293 raft_consensus.cc:348] T eee0ea2ed3b441468de9f3b7b018ffdf P e016b269efd04889b423b8a8a15d3060 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6188d9fee8af43399bb2ae8edec0e864" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 45409 } } peers { permanent_uuid: "e7cb445574c842acb6024125a3c1a213" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 33263 } } peers { permanent_uuid: "e016b269efd04889b423b8a8a15d3060" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 46167 } }
I20260806 03:50:57.321604 8297 raft_consensus.cc:348] T eee0ea2ed3b441468de9f3b7b018ffdf P 6188d9fee8af43399bb2ae8edec0e864 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6188d9fee8af43399bb2ae8edec0e864" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 45409 } } peers { permanent_uuid: "e7cb445574c842acb6024125a3c1a213" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 33263 } } peers { permanent_uuid: "e016b269efd04889b423b8a8a15d3060" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 46167 } }
I20260806 03:50:57.321770 8297 raft_consensus.cc:374] T eee0ea2ed3b441468de9f3b7b018ffdf P 6188d9fee8af43399bb2ae8edec0e864 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:50:57.321770 8293 raft_consensus.cc:374] T eee0ea2ed3b441468de9f3b7b018ffdf P e016b269efd04889b423b8a8a15d3060 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:50:57.321807 8293 raft_consensus.cc:729] T eee0ea2ed3b441468de9f3b7b018ffdf P e016b269efd04889b423b8a8a15d3060 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e016b269efd04889b423b8a8a15d3060, State: Initialized, Role: FOLLOWER
I20260806 03:50:57.321807 8297 raft_consensus.cc:729] T eee0ea2ed3b441468de9f3b7b018ffdf P 6188d9fee8af43399bb2ae8edec0e864 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6188d9fee8af43399bb2ae8edec0e864, State: Initialized, Role: FOLLOWER
I20260806 03:50:57.321967 8293 consensus_queue.cc:260] T eee0ea2ed3b441468de9f3b7b018ffdf P e016b269efd04889b423b8a8a15d3060 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6188d9fee8af43399bb2ae8edec0e864" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 45409 } } peers { permanent_uuid: "e7cb445574c842acb6024125a3c1a213" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 33263 } } peers { permanent_uuid: "e016b269efd04889b423b8a8a15d3060" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 46167 } }
I20260806 03:50:57.321967 8297 consensus_queue.cc:260] T eee0ea2ed3b441468de9f3b7b018ffdf P 6188d9fee8af43399bb2ae8edec0e864 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6188d9fee8af43399bb2ae8edec0e864" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 45409 } } peers { permanent_uuid: "e7cb445574c842acb6024125a3c1a213" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 33263 } } peers { permanent_uuid: "e016b269efd04889b423b8a8a15d3060" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 46167 } }
I20260806 03:50:57.322239 8297 ts_tablet_manager.cc:1428] T eee0ea2ed3b441468de9f3b7b018ffdf P 6188d9fee8af43399bb2ae8edec0e864: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260806 03:50:57.322242 8293 ts_tablet_manager.cc:1428] T eee0ea2ed3b441468de9f3b7b018ffdf P e016b269efd04889b423b8a8a15d3060: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260806 03:50:57.322368 8274 heartbeater.cc:503] Master 127.11.253.254:38071 was elected leader, sending a full tablet report...
I20260806 03:50:57.322362 7909 heartbeater.cc:503] Master 127.11.253.254:38071 was elected leader, sending a full tablet report...
W20260806 03:50:57.402626 8091 tablet.cc:2369] T eee0ea2ed3b441468de9f3b7b018ffdf P e7cb445574c842acb6024125a3c1a213: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 03:50:57.519657 7910 tablet.cc:2369] T eee0ea2ed3b441468de9f3b7b018ffdf P 6188d9fee8af43399bb2ae8edec0e864: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 03:50:57.531363 8276 tablet.cc:2369] T eee0ea2ed3b441468de9f3b7b018ffdf P e016b269efd04889b423b8a8a15d3060: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 03:50:57.572853 8303 raft_consensus.cc:482] T eee0ea2ed3b441468de9f3b7b018ffdf P 6188d9fee8af43399bb2ae8edec0e864 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 03:50:57.573067 8303 raft_consensus.cc:504] T eee0ea2ed3b441468de9f3b7b018ffdf P 6188d9fee8af43399bb2ae8edec0e864 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6188d9fee8af43399bb2ae8edec0e864" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 45409 } } peers { permanent_uuid: "e7cb445574c842acb6024125a3c1a213" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 33263 } } peers { permanent_uuid: "e016b269efd04889b423b8a8a15d3060" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 46167 } }
I20260806 03:50:57.573591 8303 leader_election.cc:288] T eee0ea2ed3b441468de9f3b7b018ffdf P 6188d9fee8af43399bb2ae8edec0e864 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e7cb445574c842acb6024125a3c1a213 (127.11.253.194:33263), e016b269efd04889b423b8a8a15d3060 (127.11.253.195:46167)
I20260806 03:50:57.576736 8289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:50:57.573988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.253.193:46413 (local address 127.11.253.194:33263)
0806 03:50:57.574064 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0806 03:50:57.574066 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:50:57.574079 (+ 13us) server_negotiation.cc:407] Connection header received
0806 03:50:57.574155 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:50:57.574155 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:50:57.574165 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:50:57.574299 (+ 134us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 03:50:57.574471 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:50:57.574935 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:50:57.576002 (+ 1067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:50:57.576381 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:50:57.576470 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:50:57.576479 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0806 03:50:57.576636 (+ 157us) server_negotiation.cc:299] Negotiation successful
0806 03:50:57.576671 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260806 03:50:57.576841 8290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:50:57.574072 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.253.193:40603 (local address 127.11.253.195:46167)
0806 03:50:57.574123 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0806 03:50:57.574125 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:50:57.574135 (+ 10us) server_negotiation.cc:407] Connection header received
0806 03:50:57.574165 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:50:57.574166 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:50:57.574183 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:50:57.574272 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 03:50:57.574459 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:50:57.574976 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:50:57.576034 (+ 1058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:50:57.576430 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:50:57.576550 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:50:57.576559 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0806 03:50:57.576722 (+ 163us) server_negotiation.cc:299] Negotiation successful
0806 03:50:57.576764 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260806 03:50:57.576763 8306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:50:57.573844 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.194:33263 (local address 127.11.253.193:46413)
0806 03:50:57.573984 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0806 03:50:57.573997 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 03:50:57.574087 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:50:57.574303 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:50:57.574304 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:50:57.574306 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 03:50:57.574397 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:50:57.574398 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:50:57.575024 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:50:57.575025 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:50:57.575899 (+ 874us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:50:57.575902 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:50:57.576471 (+ 569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:50:57.576472 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:50:57.576548 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:50:57.576554 (+ 6us) client_negotiation.cc:769] Sending connection context
0806 03:50:57.576617 (+ 63us) client_negotiation.cc:240] Negotiation successful
0806 03:50:57.576660 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"mutex_wait_us":47,"thread_start_us":87,"threads_started":1}
I20260806 03:50:57.576880 8307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:50:57.573846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.195:46167 (local address 127.11.253.193:40603)
0806 03:50:57.574011 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0806 03:50:57.574022 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 03:50:57.574088 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:50:57.574286 (+ 198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:50:57.574287 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:50:57.574289 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 03:50:57.574375 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:50:57.574377 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:50:57.575069 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:50:57.575070 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:50:57.575925 (+ 855us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:50:57.575927 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:50:57.576557 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:50:57.576559 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:50:57.576637 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:50:57.576641 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 03:50:57.576716 (+ 75us) client_negotiation.cc:240] Negotiation successful
0806 03:50:57.576766 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"mutex_wait_us":3,"thread_start_us":87,"threads_started":1}
I20260806 03:50:57.577387 8206 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eee0ea2ed3b441468de9f3b7b018ffdf" candidate_uuid: "6188d9fee8af43399bb2ae8edec0e864" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e016b269efd04889b423b8a8a15d3060" is_pre_election: true
I20260806 03:50:57.577385 8022 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eee0ea2ed3b441468de9f3b7b018ffdf" candidate_uuid: "6188d9fee8af43399bb2ae8edec0e864" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7cb445574c842acb6024125a3c1a213" is_pre_election: true
I20260806 03:50:57.577549 8206 raft_consensus.cc:2454] T eee0ea2ed3b441468de9f3b7b018ffdf P e016b269efd04889b423b8a8a15d3060 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6188d9fee8af43399bb2ae8edec0e864 in term 0.
I20260806 03:50:57.577549 8022 raft_consensus.cc:2454] T eee0ea2ed3b441468de9f3b7b018ffdf P e7cb445574c842acb6024125a3c1a213 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6188d9fee8af43399bb2ae8edec0e864 in term 0.
I20260806 03:50:57.577836 7766 leader_election.cc:302] T eee0ea2ed3b441468de9f3b7b018ffdf P 6188d9fee8af43399bb2ae8edec0e864 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6188d9fee8af43399bb2ae8edec0e864, e7cb445574c842acb6024125a3c1a213; no voters:
I20260806 03:50:57.578016 8303 raft_consensus.cc:2781] T eee0ea2ed3b441468de9f3b7b018ffdf P 6188d9fee8af43399bb2ae8edec0e864 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 03:50:57.578087 8303 raft_consensus.cc:482] T eee0ea2ed3b441468de9f3b7b018ffdf P 6188d9fee8af43399bb2ae8edec0e864 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 03:50:57.578106 8303 raft_consensus.cc:3037] T eee0ea2ed3b441468de9f3b7b018ffdf P 6188d9fee8af43399bb2ae8edec0e864 [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:50:57.578935 8303 raft_consensus.cc:504] T eee0ea2ed3b441468de9f3b7b018ffdf P 6188d9fee8af43399bb2ae8edec0e864 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6188d9fee8af43399bb2ae8edec0e864" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 45409 } } peers { permanent_uuid: "e7cb445574c842acb6024125a3c1a213" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 33263 } } peers { permanent_uuid: "e016b269efd04889b423b8a8a15d3060" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 46167 } }
I20260806 03:50:57.579095 8303 leader_election.cc:288] T eee0ea2ed3b441468de9f3b7b018ffdf P 6188d9fee8af43399bb2ae8edec0e864 [CANDIDATE]: Term 1 election: Requested vote from peers e7cb445574c842acb6024125a3c1a213 (127.11.253.194:33263), e016b269efd04889b423b8a8a15d3060 (127.11.253.195:46167)
I20260806 03:50:57.579298 8022 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eee0ea2ed3b441468de9f3b7b018ffdf" candidate_uuid: "6188d9fee8af43399bb2ae8edec0e864" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7cb445574c842acb6024125a3c1a213"
I20260806 03:50:57.579386 8022 raft_consensus.cc:3037] T eee0ea2ed3b441468de9f3b7b018ffdf P e7cb445574c842acb6024125a3c1a213 [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:50:57.579416 8206 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eee0ea2ed3b441468de9f3b7b018ffdf" candidate_uuid: "6188d9fee8af43399bb2ae8edec0e864" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e016b269efd04889b423b8a8a15d3060"
I20260806 03:50:57.579530 8206 raft_consensus.cc:3037] T eee0ea2ed3b441468de9f3b7b018ffdf P e016b269efd04889b423b8a8a15d3060 [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:50:57.580228 8022 raft_consensus.cc:2454] T eee0ea2ed3b441468de9f3b7b018ffdf P e7cb445574c842acb6024125a3c1a213 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6188d9fee8af43399bb2ae8edec0e864 in term 1.
I20260806 03:50:57.580422 7766 leader_election.cc:302] T eee0ea2ed3b441468de9f3b7b018ffdf P 6188d9fee8af43399bb2ae8edec0e864 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6188d9fee8af43399bb2ae8edec0e864, e7cb445574c842acb6024125a3c1a213; no voters:
I20260806 03:50:57.580440 8206 raft_consensus.cc:2454] T eee0ea2ed3b441468de9f3b7b018ffdf P e016b269efd04889b423b8a8a15d3060 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6188d9fee8af43399bb2ae8edec0e864 in term 1.
I20260806 03:50:57.580580 8303 raft_consensus.cc:2781] T eee0ea2ed3b441468de9f3b7b018ffdf P 6188d9fee8af43399bb2ae8edec0e864 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 03:50:57.580766 8303 raft_consensus.cc:686] T eee0ea2ed3b441468de9f3b7b018ffdf P 6188d9fee8af43399bb2ae8edec0e864 [term 1 LEADER]: Becoming Leader. State: Replica: 6188d9fee8af43399bb2ae8edec0e864, State: Running, Role: LEADER
I20260806 03:50:57.580909 8303 consensus_queue.cc:237] T eee0ea2ed3b441468de9f3b7b018ffdf P 6188d9fee8af43399bb2ae8edec0e864 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6188d9fee8af43399bb2ae8edec0e864" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 45409 } } peers { permanent_uuid: "e7cb445574c842acb6024125a3c1a213" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 33263 } } peers { permanent_uuid: "e016b269efd04889b423b8a8a15d3060" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 46167 } }
I20260806 03:50:57.581998 7667 catalog_manager.cc:5495] T eee0ea2ed3b441468de9f3b7b018ffdf P 6188d9fee8af43399bb2ae8edec0e864 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6188d9fee8af43399bb2ae8edec0e864 (127.11.253.193). New cstate: current_term: 1 leader_uuid: "6188d9fee8af43399bb2ae8edec0e864" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6188d9fee8af43399bb2ae8edec0e864" member_type: VOTER last_known_addr { host: "127.11.253.193" port: 45409 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e7cb445574c842acb6024125a3c1a213" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 33263 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e016b269efd04889b423b8a8a15d3060" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 46167 } health_report { overall_health: UNKNOWN } } }
I20260806 03:50:57.593591 8292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:50:57.591100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33404 (local address 127.11.253.193:45409)
0806 03:50:57.591163 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0806 03:50:57.591165 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:50:57.591174 (+ 9us) server_negotiation.cc:407] Connection header received
0806 03:50:57.591282 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:50:57.591283 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:50:57.591310 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:50:57.591389 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 03:50:57.591518 (+ 129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:50:57.591977 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:50:57.592794 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:50:57.592991 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:50:57.593076 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:50:57.593241 (+ 165us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 03:50:57.593337 (+ 96us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 03:50:57.593422 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0806 03:50:57.593488 (+ 66us) server_negotiation.cc:299] Negotiation successful
0806 03:50:57.593508 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260806 03:50:57.595391 8206 raft_consensus.cc:1261] T eee0ea2ed3b441468de9f3b7b018ffdf P e016b269efd04889b423b8a8a15d3060 [term 1 FOLLOWER]: Refusing update from remote peer 6188d9fee8af43399bb2ae8edec0e864: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 03:50:57.595431 8022 raft_consensus.cc:1261] T eee0ea2ed3b441468de9f3b7b018ffdf P e7cb445574c842acb6024125a3c1a213 [term 1 FOLLOWER]: Refusing update from remote peer 6188d9fee8af43399bb2ae8edec0e864: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 03:50:57.595887 8308 consensus_queue.cc:1035] T eee0ea2ed3b441468de9f3b7b018ffdf P 6188d9fee8af43399bb2ae8edec0e864 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e016b269efd04889b423b8a8a15d3060" member_type: VOTER last_known_addr { host: "127.11.253.195" port: 46167 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 03:50:57.596032 8303 consensus_queue.cc:1035] T eee0ea2ed3b441468de9f3b7b018ffdf P 6188d9fee8af43399bb2ae8edec0e864 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7cb445574c842acb6024125a3c1a213" member_type: VOTER last_known_addr { host: "127.11.253.194" port: 33263 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 03:50:57.598489 8314 mvcc.cc:205] Tried to move back new op lower bound from 7315407903108153344 to 7315407903052337152. Current Snapshot: MvccSnapshot[applied={T|T < 7315407903108153344}]
I20260806 03:50:57.598841 8315 mvcc.cc:205] Tried to move back new op lower bound from 7315407903108153344 to 7315407903052337152. Current Snapshot: MvccSnapshot[applied={T|T < 7315407903108153344}]
I20260806 03:50:57.599095 8318 mvcc.cc:205] Tried to move back new op lower bound from 7315407903108153344 to 7315407903052337152. Current Snapshot: MvccSnapshot[applied={T|T < 7315407903108153344}]
I20260806 03:50:57.600744 7668 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:40146:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 03:50:57.600833 7668 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:40146:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 03:50:57.601653 7668 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 736b3a5168094436a61cdb4f74ec0958: Sending DeleteTablet for 3 replicas of tablet eee0ea2ed3b441468de9f3b7b018ffdf
I20260806 03:50:57.602023 7827 tablet_service.cc:1514] Processing DeleteTablet for tablet eee0ea2ed3b441468de9f3b7b018ffdf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 03:50:57 UTC) from {username='slave'} at 127.0.0.1:33392
I20260806 03:50:57.602051 8184 tablet_service.cc:1514] Processing DeleteTablet for tablet eee0ea2ed3b441468de9f3b7b018ffdf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 03:50:57 UTC) from {username='slave'} at 127.0.0.1:38114
I20260806 03:50:57.602221 7997 tablet_service.cc:1514] Processing DeleteTablet for tablet eee0ea2ed3b441468de9f3b7b018ffdf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 03:50:57 UTC) from {username='slave'} at 127.0.0.1:59924
I20260806 03:50:57.602368 8326 tablet_replica.cc:330] T eee0ea2ed3b441468de9f3b7b018ffdf P e016b269efd04889b423b8a8a15d3060: stopping tablet replica
I20260806 03:50:57.602481 8326 raft_consensus.cc:2229] T eee0ea2ed3b441468de9f3b7b018ffdf P e016b269efd04889b423b8a8a15d3060 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 03:50:57.602509 8328 tablet_replica.cc:330] T eee0ea2ed3b441468de9f3b7b018ffdf P e7cb445574c842acb6024125a3c1a213: stopping tablet replica
I20260806 03:50:57.602593 8326 raft_consensus.cc:2258] T eee0ea2ed3b441468de9f3b7b018ffdf P e016b269efd04889b423b8a8a15d3060 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 03:50:57.602723 8328 raft_consensus.cc:2229] T eee0ea2ed3b441468de9f3b7b018ffdf P e7cb445574c842acb6024125a3c1a213 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 03:50:57.602800 12279 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskToVWRe/build/release/bin/kudu with pid 7744
I20260806 03:50:57.602958 8328 raft_consensus.cc:2258] T eee0ea2ed3b441468de9f3b7b018ffdf P e7cb445574c842acb6024125a3c1a213 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 03:50:57.603016 8326 ts_tablet_manager.cc:1905] T eee0ea2ed3b441468de9f3b7b018ffdf P e016b269efd04889b423b8a8a15d3060: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 03:50:57.606956 8326 ts_tablet_manager.cc:1918] T eee0ea2ed3b441468de9f3b7b018ffdf P e016b269efd04889b423b8a8a15d3060: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 03:50:57.607044 8326 log.cc:1196] T eee0ea2ed3b441468de9f3b7b018ffdf P e016b269efd04889b423b8a8a15d3060: Deleting WAL directory at /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785988004637995-12279-0/minicluster-data/ts-2/wal/wals/eee0ea2ed3b441468de9f3b7b018ffdf
I20260806 03:50:57.607297 8326 ts_tablet_manager.cc:1939] T eee0ea2ed3b441468de9f3b7b018ffdf P e016b269efd04889b423b8a8a15d3060: Deleting consensus metadata
I20260806 03:50:57.607831 7652 catalog_manager.cc:4841] TS e016b269efd04889b423b8a8a15d3060 (127.11.253.195:46167): tablet eee0ea2ed3b441468de9f3b7b018ffdf (table test-table [id=12faa6467b0f467caa1514301e5beac0]) successfully deleted
I20260806 03:50:57.607942 8328 ts_tablet_manager.cc:1905] T eee0ea2ed3b441468de9f3b7b018ffdf P e7cb445574c842acb6024125a3c1a213: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 03:50:57.610288 8328 ts_tablet_manager.cc:1918] T eee0ea2ed3b441468de9f3b7b018ffdf P e7cb445574c842acb6024125a3c1a213: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 03:50:57.610400 8328 log.cc:1196] T eee0ea2ed3b441468de9f3b7b018ffdf P e7cb445574c842acb6024125a3c1a213: Deleting WAL directory at /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785988004637995-12279-0/minicluster-data/ts-1/wal/wals/eee0ea2ed3b441468de9f3b7b018ffdf
I20260806 03:50:57.610661 8328 ts_tablet_manager.cc:1939] T eee0ea2ed3b441468de9f3b7b018ffdf P e7cb445574c842acb6024125a3c1a213: Deleting consensus metadata
I20260806 03:50:57.611145 7655 catalog_manager.cc:4841] TS e7cb445574c842acb6024125a3c1a213 (127.11.253.194:33263): tablet eee0ea2ed3b441468de9f3b7b018ffdf (table test-table [id=12faa6467b0f467caa1514301e5beac0]) successfully deleted
W20260806 03:50:57.614688 7654 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.253.193:45409 (error 108)
I20260806 03:50:57.614940 12279 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskToVWRe/build/release/bin/kudu with pid 7915
I20260806 03:50:57.618589 8284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:50:57.618231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.193:45409 (local address 127.0.0.1:33414)
0806 03:50:57.618475 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0806 03:50:57.618542 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.253.193:45409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":212}
W20260806 03:50:57.619314 7654 catalog_manager.cc:4584] TS 6188d9fee8af43399bb2ae8edec0e864 (127.11.253.193:45409): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet eee0ea2ed3b441468de9f3b7b018ffdf: Network error: Client connection negotiation failed: client connection to 127.11.253.193:45409: connect: Connection refused (error 111)
I20260806 03:50:57.629604 12279 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskToVWRe/build/release/bin/kudu with pid 8094
I20260806 03:50:57.640311 12279 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskToVWRe/build/release/bin/kudu with pid 7626
2026-08-06T03:50:57Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (945 ms)
[----------] 6 tests from EncryptionPolicyTest (9392 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785988004637995-12279-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 03:50:57 dist-test-slave-6ntk krb5kdc[8333](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 03:50:57 dist-test-slave-6ntk krb5kdc[8333](info): set up 2 sockets
Aug 06 03:50:57 dist-test-slave-6ntk krb5kdc[8333](info): commencing operation
krb5kdc: starting...
W20260806 03:50:58.849081 12279 mini_kdc.cc:121] Time spent starting KDC: real 1.183s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785988004637995-12279-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785988004637995-12279-0/krb5kdc/test-user.keytab.
Aug 06 03:50:58 dist-test-slave-6ntk krb5kdc[8333](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988258, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T03:50:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T03:50:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785988004637995-12279-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785988004637995-12279-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 03:50:58.985203 12279 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskToVWRe/build/release/bin/kudu
/tmp/dist-test-taskToVWRe/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785988004637995-12279-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785988004637995-12279-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785988004637995-12279-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785988004637995-12279-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43801
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785988004637995-12279-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37505
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43801
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785988004637995-12279-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785988004637995-12279-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785988004637995-12279-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:50:59.071564 8350 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:50:59.071815 8350 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:50:59.071856 8350 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:50:59.073050 8350 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 03:50:59.073127 8350 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:50:59.073153 8350 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:50:59.073181 8350 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 03:50:59.073213 8350 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 03:50:59.074769 8350 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37505
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785988004637995-12279-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785988004637995-12279-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43801
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785988004637995-12279-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43801
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785988004637995-12279-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8350
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785988004637995-12279-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:50:59.075059 8350 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:50:59.075304 8350 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:50:59.080639 8356 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:50:59.080652 8360 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:50:59.080627 8358 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:50:59.081141 8350 server_base.cc:1034] running on GCE node
I20260806 03:50:59.081429 8350 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:50:59.081717 8350 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:50:59.082882 8350 hybrid_clock.cc:648] HybridClock initialized: now 1785988259082864 us; error 50 us; skew 500 ppm
Aug 06 03:50:59 dist-test-slave-6ntk krb5kdc[8333](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988259, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:50:59.090167 8350 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 03:50:59.090745 8350 webserver.cc:460] Webserver started at http://127.0.0.1:41263/ using document root <none> and password file <none>
I20260806 03:50:59.090986 8350 fs_manager.cc:359] Metadata directory not provided
I20260806 03:50:59.091058 8350 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:50:59.091217 8350 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:50:59.092340 8350 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785988004637995-12279-0/minicluster-data/master-0/data/instance:
uuid: "71dd25bfee494808befbc89f1fe2bd79"
format_stamp: "Formatted at 2026-08-06 03:50:59 on dist-test-slave-6ntk"
server_key: "15aa4779e257ab9460936087bb58046f"
server_key_iv: "57d209fb682827878a2d1e65e73cc167"
server_key_version: "encryptionkey@0"
I20260806 03:50:59.092724 8350 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785988004637995-12279-0/minicluster-data/master-0/wal/instance:
uuid: "71dd25bfee494808befbc89f1fe2bd79"
format_stamp: "Formatted at 2026-08-06 03:50:59 on dist-test-slave-6ntk"
server_key: "15aa4779e257ab9460936087bb58046f"
server_key_iv: "57d209fb682827878a2d1e65e73cc167"
server_key_version: "encryptionkey@0"
I20260806 03:50:59.094197 8350 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260806 03:50:59.094995 8368 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:50:59.095167 8350 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 03:50:59.095319 8350 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785988004637995-12279-0/minicluster-data/master-0/data,/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785988004637995-12279-0/minicluster-data/master-0/wal
uuid: "71dd25bfee494808befbc89f1fe2bd79"
format_stamp: "Formatted at 2026-08-06 03:50:59 on dist-test-slave-6ntk"
server_key: "15aa4779e257ab9460936087bb58046f"
server_key_iv: "57d209fb682827878a2d1e65e73cc167"
server_key_version: "encryptionkey@0"
I20260806 03:50:59.095403 8350 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785988004637995-12279-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785988004637995-12279-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785988004637995-12279-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:50:59.112363 8350 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:50:59.114878 8350 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:50:59.115054 8350 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:50:59.119519 8350 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43801
I20260806 03:50:59.119555 8434 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43801 every 8 connection(s)
I20260806 03:50:59.119938 8350 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785988004637995-12279-0/minicluster-data/master-0/data/info.pb
I20260806 03:50:59.120875 8436 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:50:59.121474 12279 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskToVWRe/build/release/bin/kudu as pid 8350
I20260806 03:50:59.121598 12279 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785988004637995-12279-0/minicluster-data/master-0/wal/instance
I20260806 03:50:59.121794 12279 external_mini_cluster.cc:1428] Setting key 3f806d53c87d81be4ab94aad91722e45
I20260806 03:50:59.124153 8436 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 71dd25bfee494808befbc89f1fe2bd79: Bootstrap starting.
I20260806 03:50:59.124804 8436 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 71dd25bfee494808befbc89f1fe2bd79: Neither blocks nor log segments found. Creating new log.
I20260806 03:50:59.125114 8436 log.cc:824] T 00000000000000000000000000000000 P 71dd25bfee494808befbc89f1fe2bd79: Log is configured to *not* fsync() on all Append() calls
I20260806 03:50:59.125937 8436 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 71dd25bfee494808befbc89f1fe2bd79: No bootstrap required, opened a new log
I20260806 03:50:59.128538 8436 raft_consensus.cc:348] T 00000000000000000000000000000000 P 71dd25bfee494808befbc89f1fe2bd79 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "71dd25bfee494808befbc89f1fe2bd79" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43801 } }
I20260806 03:50:59.128681 8436 raft_consensus.cc:374] T 00000000000000000000000000000000 P 71dd25bfee494808befbc89f1fe2bd79 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:50:59.128705 8436 raft_consensus.cc:729] T 00000000000000000000000000000000 P 71dd25bfee494808befbc89f1fe2bd79 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 71dd25bfee494808befbc89f1fe2bd79, State: Initialized, Role: FOLLOWER
I20260806 03:50:59.128849 8436 consensus_queue.cc:260] T 00000000000000000000000000000000 P 71dd25bfee494808befbc89f1fe2bd79 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "71dd25bfee494808befbc89f1fe2bd79" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43801 } }
I20260806 03:50:59.128929 8436 raft_consensus.cc:388] T 00000000000000000000000000000000 P 71dd25bfee494808befbc89f1fe2bd79 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 03:50:59.128954 8436 raft_consensus.cc:482] T 00000000000000000000000000000000 P 71dd25bfee494808befbc89f1fe2bd79 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 03:50:59.129032 8436 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 71dd25bfee494808befbc89f1fe2bd79 [term 0 FOLLOWER]: Advancing to term 1
Aug 06 03:50:59 dist-test-slave-6ntk krb5kdc[8333](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988258, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260806 03:50:59.129681 8436 raft_consensus.cc:504] T 00000000000000000000000000000000 P 71dd25bfee494808befbc89f1fe2bd79 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "71dd25bfee494808befbc89f1fe2bd79" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43801 } }
I20260806 03:50:59.129827 8436 leader_election.cc:302] T 00000000000000000000000000000000 P 71dd25bfee494808befbc89f1fe2bd79 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 71dd25bfee494808befbc89f1fe2bd79; no voters:
I20260806 03:50:59.130038 8436 leader_election.cc:288] T 00000000000000000000000000000000 P 71dd25bfee494808befbc89f1fe2bd79 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 03:50:59.130072 8443 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 71dd25bfee494808befbc89f1fe2bd79 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 03:50:59.130193 8443 raft_consensus.cc:686] T 00000000000000000000000000000000 P 71dd25bfee494808befbc89f1fe2bd79 [term 1 LEADER]: Becoming Leader. State: Replica: 71dd25bfee494808befbc89f1fe2bd79, State: Running, Role: LEADER
I20260806 03:50:59.130287 8443 consensus_queue.cc:237] T 00000000000000000000000000000000 P 71dd25bfee494808befbc89f1fe2bd79 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "71dd25bfee494808befbc89f1fe2bd79" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43801 } }
I20260806 03:50:59.130342 8436 sys_catalog.cc:564] T 00000000000000000000000000000000 P 71dd25bfee494808befbc89f1fe2bd79 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 03:50:59.130677 8443 sys_catalog.cc:455] T 00000000000000000000000000000000 P 71dd25bfee494808befbc89f1fe2bd79 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 71dd25bfee494808befbc89f1fe2bd79. Latest consensus state: current_term: 1 leader_uuid: "71dd25bfee494808befbc89f1fe2bd79" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "71dd25bfee494808befbc89f1fe2bd79" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43801 } } }
I20260806 03:50:59.130774 8443 sys_catalog.cc:458] T 00000000000000000000000000000000 P 71dd25bfee494808befbc89f1fe2bd79 [sys.catalog]: This master's current role is: LEADER
I20260806 03:50:59.130689 8447 sys_catalog.cc:455] T 00000000000000000000000000000000 P 71dd25bfee494808befbc89f1fe2bd79 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "71dd25bfee494808befbc89f1fe2bd79" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "71dd25bfee494808befbc89f1fe2bd79" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43801 } } }
I20260806 03:50:59.131090 8447 sys_catalog.cc:458] T 00000000000000000000000000000000 P 71dd25bfee494808befbc89f1fe2bd79 [sys.catalog]: This master's current role is: LEADER
I20260806 03:50:59.131721 8452 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 03:50:59.132652 8452 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 03:50:59.133808 8439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:50:59.122440 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41119 (local address 127.0.0.1:43801)
0806 03:50:59.122608 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0806 03:50:59.122618 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 03:50:59.122652 (+ 34us) server_negotiation.cc:407] Connection header received
0806 03:50:59.123063 (+ 411us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:50:59.123069 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:50:59.123116 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:50:59.123250 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:50:59.124089 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:50:59.125109 (+ 1020us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:50:59.126171 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:50:59.126585 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:50:59.126668 (+ 83us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:50:59.129844 (+ 3176us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:50:59.129850 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:50:59.129850 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:50:59.129863 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:50:59.132090 (+ 2227us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:50:59.132538 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:50:59.132542 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:50:59.132542 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:50:59.132641 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:50:59.132925 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:50:59.132927 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:50:59.132927 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:50:59.133180 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:50:59.133367 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0806 03:50:59.133485 (+ 118us) server_negotiation.cc:299] Negotiation successful
0806 03:50:59.133592 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":82,"threads_started":1}
I20260806 03:50:59.134404 8452 catalog_manager.cc:1347] Generated new cluster ID: ffd95cd614f64fbcb6d1e8cfb16f6faf
I20260806 03:50:59.134454 8452 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 03:50:59.134909 12279 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 03:50:59.139990 8452 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 03:50:59.140679 8452 catalog_manager.cc:1504] Loading token signing keys...
I20260806 03:50:59.144812 8439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:50:59.136335 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46703 (local address 127.0.0.1:43801)
0806 03:50:59.136366 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0806 03:50:59.136368 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:50:59.138090 (+ 1722us) server_negotiation.cc:407] Connection header received
0806 03:50:59.138122 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:50:59.138123 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:50:59.138134 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:50:59.138215 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:50:59.138835 (+ 620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:50:59.139586 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:50:59.140465 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:50:59.140713 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:50:59.140797 (+ 84us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:50:59.142133 (+ 1336us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:50:59.142134 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:50:59.142134 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:50:59.142141 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:50:59.143618 (+ 1477us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:50:59.143999 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:50:59.144000 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:50:59.144001 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:50:59.144060 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:50:59.144296 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:50:59.144296 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:50:59.144297 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:50:59.144407 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:50:59.144538 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0806 03:50:59.144647 (+ 109us) server_negotiation.cc:299] Negotiation successful
0806 03:50:59.144712 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260806 03:50:59.151394 8452 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 71dd25bfee494808befbc89f1fe2bd79: Generated new TSK 0
I20260806 03:50:59.151580 8452 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 03:50:59.155478 12279 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskToVWRe/build/release/bin/kudu with pid 8350
W20260806 03:50:59.165658 8330 connection.cc:535] client connection to 127.0.0.1:43801 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:43801): Connection reset by peer (error 104)
2026-08-06T03:50:59Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1513 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785988004637995-12279-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 03:50:59 dist-test-slave-6ntk krb5kdc[8482](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 03:50:59 dist-test-slave-6ntk krb5kdc[8482](info): set up 2 sockets
Aug 06 03:50:59 dist-test-slave-6ntk krb5kdc[8482](info): commencing operation
krb5kdc: starting...
W20260806 03:51:00.364615 12279 mini_kdc.cc:121] Time spent starting KDC: real 1.185s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785988004637995-12279-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785988004637995-12279-0/krb5kdc/test-user.keytab.
Aug 06 03:51:00 dist-test-slave-6ntk krb5kdc[8482](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988260, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T03:51:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T03:51:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785988004637995-12279-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785988004637995-12279-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 03:51:00.502492 12279 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskToVWRe/build/release/bin/kudu
/tmp/dist-test-taskToVWRe/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785988004637995-12279-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785988004637995-12279-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785988004637995-12279-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785988004637995-12279-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46823
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785988004637995-12279-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34611
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46823
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785988004637995-12279-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785988004637995-12279-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785988004637995-12279-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:51:00.589035 8499 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:51:00.589263 8499 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:51:00.589287 8499 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:51:00.590667 8499 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 03:51:00.590721 8499 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:51:00.590736 8499 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:51:00.590782 8499 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 03:51:00.590808 8499 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 03:51:00.592368 8499 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34611
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785988004637995-12279-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785988004637995-12279-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46823
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785988004637995-12279-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46823
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785988004637995-12279-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8499
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785988004637995-12279-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:51:00.592656 8499 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:51:00.592906 8499 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:51:00.598146 8507 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:51:00.598135 8511 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:51:00.598135 8508 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:51:00.598456 8499 server_base.cc:1034] running on GCE node
I20260806 03:51:00.598718 8499 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:51:00.598981 8499 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:51:00.600157 8499 hybrid_clock.cc:648] HybridClock initialized: now 1785988260600142 us; error 30 us; skew 500 ppm
Aug 06 03:51:00 dist-test-slave-6ntk krb5kdc[8482](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988260, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:51:00.603010 8499 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 03:51:00.603514 8499 webserver.cc:460] Webserver started at http://127.0.0.1:44315/ using document root <none> and password file <none>
I20260806 03:51:00.603753 8499 fs_manager.cc:359] Metadata directory not provided
I20260806 03:51:00.603808 8499 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:51:00.603943 8499 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:51:00.605055 8499 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785988004637995-12279-0/minicluster-data/master-0/data/instance:
uuid: "4a35af7362094275a0c677521c6f4f59"
format_stamp: "Formatted at 2026-08-06 03:51:00 on dist-test-slave-6ntk"
server_key: "baef2330333f46e5856ebcdacbab9276"
server_key_iv: "4ec8d8c1c314df1475da12239c1d5e4e"
server_key_version: "encryptionkey@0"
I20260806 03:51:00.605448 8499 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785988004637995-12279-0/minicluster-data/master-0/wal/instance:
uuid: "4a35af7362094275a0c677521c6f4f59"
format_stamp: "Formatted at 2026-08-06 03:51:00 on dist-test-slave-6ntk"
server_key: "baef2330333f46e5856ebcdacbab9276"
server_key_iv: "4ec8d8c1c314df1475da12239c1d5e4e"
server_key_version: "encryptionkey@0"
I20260806 03:51:00.607040 8499 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 03:51:00.607892 8521 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:51:00.608114 8499 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260806 03:51:00.608196 8499 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785988004637995-12279-0/minicluster-data/master-0/data,/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785988004637995-12279-0/minicluster-data/master-0/wal
uuid: "4a35af7362094275a0c677521c6f4f59"
format_stamp: "Formatted at 2026-08-06 03:51:00 on dist-test-slave-6ntk"
server_key: "baef2330333f46e5856ebcdacbab9276"
server_key_iv: "4ec8d8c1c314df1475da12239c1d5e4e"
server_key_version: "encryptionkey@0"
I20260806 03:51:00.608290 8499 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785988004637995-12279-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785988004637995-12279-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785988004637995-12279-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:51:00.629329 8499 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:51:00.632077 8499 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:51:00.632268 8499 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:51:00.636971 8499 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46823
I20260806 03:51:00.637017 8589 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46823 every 8 connection(s)
I20260806 03:51:00.637419 8499 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785988004637995-12279-0/minicluster-data/master-0/data/info.pb
I20260806 03:51:00.638495 8592 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:51:00.638837 12279 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskToVWRe/build/release/bin/kudu as pid 8499
I20260806 03:51:00.638969 12279 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785988004637995-12279-0/minicluster-data/master-0/wal/instance
I20260806 03:51:00.639169 12279 external_mini_cluster.cc:1428] Setting key 90c5091a19156ccfaf4496f0e181b85c
I20260806 03:51:00.641392 8592 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4a35af7362094275a0c677521c6f4f59: Bootstrap starting.
I20260806 03:51:00.642146 8592 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4a35af7362094275a0c677521c6f4f59: Neither blocks nor log segments found. Creating new log.
I20260806 03:51:00.642462 8592 log.cc:824] T 00000000000000000000000000000000 P 4a35af7362094275a0c677521c6f4f59: Log is configured to *not* fsync() on all Append() calls
I20260806 03:51:00.643236 8592 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4a35af7362094275a0c677521c6f4f59: No bootstrap required, opened a new log
I20260806 03:51:00.644889 8595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:00.639811 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53717 (local address 127.0.0.1:46823)
0806 03:51:00.640022 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0806 03:51:00.640031 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 03:51:00.640058 (+ 27us) server_negotiation.cc:407] Connection header received
0806 03:51:00.640486 (+ 428us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:51:00.640492 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:51:00.640539 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:51:00.640702 (+ 163us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:51:00.640963 (+ 261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:00.641977 (+ 1014us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:00.642912 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:00.643387 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:00.643514 (+ 127us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:00.644263 (+ 749us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:51:00.644268 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:51:00.644268 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 03:51:00.644453 (+ 185us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:51:00.644484 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 03:51:00.644495 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:51:00.644591 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0806 03:51:00.644674 (+ 83us) server_negotiation.cc:299] Negotiation successful
0806 03:51:00.644705 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":159,"thread_start_us":93,"threads_started":1}
I20260806 03:51:00.645354 8592 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4a35af7362094275a0c677521c6f4f59 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a35af7362094275a0c677521c6f4f59" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46823 } }
I20260806 03:51:00.645565 8592 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4a35af7362094275a0c677521c6f4f59 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:51:00.645627 8592 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4a35af7362094275a0c677521c6f4f59 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4a35af7362094275a0c677521c6f4f59, State: Initialized, Role: FOLLOWER
I20260806 03:51:00.645782 8592 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4a35af7362094275a0c677521c6f4f59 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a35af7362094275a0c677521c6f4f59" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46823 } }
I20260806 03:51:00.645921 8592 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4a35af7362094275a0c677521c6f4f59 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 03:51:00.646004 8592 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4a35af7362094275a0c677521c6f4f59 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 03:51:00.645998 12279 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 03:51:00.646067 8592 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4a35af7362094275a0c677521c6f4f59 [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:51:00.646903 8592 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4a35af7362094275a0c677521c6f4f59 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a35af7362094275a0c677521c6f4f59" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46823 } }
I20260806 03:51:00.647030 8592 leader_election.cc:302] T 00000000000000000000000000000000 P 4a35af7362094275a0c677521c6f4f59 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4a35af7362094275a0c677521c6f4f59; no voters:
I20260806 03:51:00.647234 8592 leader_election.cc:288] T 00000000000000000000000000000000 P 4a35af7362094275a0c677521c6f4f59 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 03:51:00.647284 8600 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4a35af7362094275a0c677521c6f4f59 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 03:51:00.647444 8600 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4a35af7362094275a0c677521c6f4f59 [term 1 LEADER]: Becoming Leader. State: Replica: 4a35af7362094275a0c677521c6f4f59, State: Running, Role: LEADER
I20260806 03:51:00.647516 8592 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4a35af7362094275a0c677521c6f4f59 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 03:51:00.647553 8600 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4a35af7362094275a0c677521c6f4f59 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a35af7362094275a0c677521c6f4f59" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46823 } }
I20260806 03:51:00.648123 8604 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4a35af7362094275a0c677521c6f4f59 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4a35af7362094275a0c677521c6f4f59" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a35af7362094275a0c677521c6f4f59" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46823 } } }
I20260806 03:51:00.648116 8600 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4a35af7362094275a0c677521c6f4f59 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4a35af7362094275a0c677521c6f4f59. Latest consensus state: current_term: 1 leader_uuid: "4a35af7362094275a0c677521c6f4f59" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a35af7362094275a0c677521c6f4f59" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46823 } } }
I20260806 03:51:00.648212 8604 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4a35af7362094275a0c677521c6f4f59 [sys.catalog]: This master's current role is: LEADER
I20260806 03:51:00.648219 8600 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4a35af7362094275a0c677521c6f4f59 [sys.catalog]: This master's current role is: LEADER
I20260806 03:51:00.648435 8608 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 03:51:00.648949 8608 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 03:51:00.650756 8608 catalog_manager.cc:1347] Generated new cluster ID: 2a43a2e01e7345298d376f669467fc34
I20260806 03:51:00.650815 8608 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 03:51:00.652062 8595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:00.648016 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40217 (local address 127.0.0.1:46823)
0806 03:51:00.648057 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0806 03:51:00.648060 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 03:51:00.648071 (+ 11us) server_negotiation.cc:407] Connection header received
0806 03:51:00.648094 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:51:00.648095 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:51:00.648105 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:51:00.648162 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:51:00.648378 (+ 216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:00.649163 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:00.650420 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:00.650749 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:00.650796 (+ 47us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:00.651744 (+ 948us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:51:00.651746 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:51:00.651746 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 03:51:00.651752 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:51:00.651768 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 03:51:00.651774 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:51:00.651812 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0806 03:51:00.651971 (+ 159us) server_negotiation.cc:299] Negotiation successful
0806 03:51:00.651986 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 03:51:00.681246 8608 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 03:51:00.682076 8608 catalog_manager.cc:1504] Loading token signing keys...
I20260806 03:51:00.685029 8608 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4a35af7362094275a0c677521c6f4f59: Generated new TSK 0
I20260806 03:51:00.685236 8608 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 03:51:00.690311 12279 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskToVWRe/build/release/bin/kudu with pid 8499
W20260806 03:51:00.697686 8479 connection.cc:535] client connection to 127.0.0.1:46823 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:46823): Connection reset by peer (error 104)
2026-08-06T03:51:00Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1532 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785988004637995-12279-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 03:51:00 dist-test-slave-6ntk krb5kdc[8627](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 03:51:00 dist-test-slave-6ntk krb5kdc[8627](info): set up 2 sockets
Aug 06 03:51:00 dist-test-slave-6ntk krb5kdc[8627](info): commencing operation
krb5kdc: starting...
W20260806 03:51:02.019970 12279 mini_kdc.cc:121] Time spent starting KDC: real 1.308s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785988004637995-12279-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785988004637995-12279-0/krb5kdc/test-user.keytab.
Aug 06 03:51:02 dist-test-slave-6ntk krb5kdc[8627](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988262, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T03:51:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T03:51:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785988004637995-12279-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785988004637995-12279-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 03:51:02.156544 12279 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskToVWRe/build/release/bin/kudu
/tmp/dist-test-taskToVWRe/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785988004637995-12279-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785988004637995-12279-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785988004637995-12279-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785988004637995-12279-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34629
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785988004637995-12279-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40607
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34629
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785988004637995-12279-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785988004637995-12279-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785988004637995-12279-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:51:02.244230 8650 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:51:02.244444 8650 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:51:02.244467 8650 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:51:02.245669 8650 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 03:51:02.245719 8650 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:51:02.245734 8650 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:51:02.245747 8650 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 03:51:02.245759 8650 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 03:51:02.247329 8650 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40607
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785988004637995-12279-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785988004637995-12279-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34629
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785988004637995-12279-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34629
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785988004637995-12279-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8650
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785988004637995-12279-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:51:02.247598 8650 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:51:02.247865 8650 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:51:02.253238 8658 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:51:02.253228 8660 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:51:02.253234 8662 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:51:02.253526 8650 server_base.cc:1034] running on GCE node
I20260806 03:51:02.253795 8650 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:51:02.254165 8650 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:51:02.255333 8650 hybrid_clock.cc:648] HybridClock initialized: now 1785988262255319 us; error 33 us; skew 500 ppm
Aug 06 03:51:02 dist-test-slave-6ntk krb5kdc[8627](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988262, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:51:02.258265 8650 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 03:51:02.258759 8650 webserver.cc:460] Webserver started at http://127.0.0.1:40359/ using document root <none> and password file <none>
I20260806 03:51:02.259011 8650 fs_manager.cc:359] Metadata directory not provided
I20260806 03:51:02.259057 8650 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:51:02.259217 8650 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:51:02.260419 8650 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785988004637995-12279-0/minicluster-data/master-0/data/instance:
uuid: "67fee47a4f5b4b02afdb9fa68b746571"
format_stamp: "Formatted at 2026-08-06 03:51:02 on dist-test-slave-6ntk"
server_key: "645f848a2defa9de053acb62d0792ed9"
server_key_iv: "6b54472d79cc888ac5bfd64d9c3356b5"
server_key_version: "encryptionkey@0"
I20260806 03:51:02.260821 8650 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785988004637995-12279-0/minicluster-data/master-0/wal/instance:
uuid: "67fee47a4f5b4b02afdb9fa68b746571"
format_stamp: "Formatted at 2026-08-06 03:51:02 on dist-test-slave-6ntk"
server_key: "645f848a2defa9de053acb62d0792ed9"
server_key_iv: "6b54472d79cc888ac5bfd64d9c3356b5"
server_key_version: "encryptionkey@0"
I20260806 03:51:02.262426 8650 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 03:51:02.263286 8673 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:51:02.263473 8650 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 03:51:02.263588 8650 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785988004637995-12279-0/minicluster-data/master-0/data,/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785988004637995-12279-0/minicluster-data/master-0/wal
uuid: "67fee47a4f5b4b02afdb9fa68b746571"
format_stamp: "Formatted at 2026-08-06 03:51:02 on dist-test-slave-6ntk"
server_key: "645f848a2defa9de053acb62d0792ed9"
server_key_iv: "6b54472d79cc888ac5bfd64d9c3356b5"
server_key_version: "encryptionkey@0"
I20260806 03:51:02.263667 8650 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785988004637995-12279-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785988004637995-12279-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785988004637995-12279-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:51:02.267247 8650 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:51:02.269723 8650 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:51:02.269959 8650 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:51:02.274289 8650 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34629
I20260806 03:51:02.274364 8737 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34629 every 8 connection(s)
I20260806 03:51:02.274709 8650 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785988004637995-12279-0/minicluster-data/master-0/data/info.pb
I20260806 03:51:02.275681 8738 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:51:02.278206 8738 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 67fee47a4f5b4b02afdb9fa68b746571: Bootstrap starting.
I20260806 03:51:02.278854 8738 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 67fee47a4f5b4b02afdb9fa68b746571: Neither blocks nor log segments found. Creating new log.
I20260806 03:51:02.279161 8738 log.cc:824] T 00000000000000000000000000000000 P 67fee47a4f5b4b02afdb9fa68b746571: Log is configured to *not* fsync() on all Append() calls
I20260806 03:51:02.280180 8738 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 67fee47a4f5b4b02afdb9fa68b746571: No bootstrap required, opened a new log
I20260806 03:51:02.282215 12279 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskToVWRe/build/release/bin/kudu as pid 8650
I20260806 03:51:02.282181 8738 raft_consensus.cc:348] T 00000000000000000000000000000000 P 67fee47a4f5b4b02afdb9fa68b746571 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "67fee47a4f5b4b02afdb9fa68b746571" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34629 } }
I20260806 03:51:02.282362 12279 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785988004637995-12279-0/minicluster-data/master-0/wal/instance
I20260806 03:51:02.282410 8738 raft_consensus.cc:374] T 00000000000000000000000000000000 P 67fee47a4f5b4b02afdb9fa68b746571 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:51:02.282472 8738 raft_consensus.cc:729] T 00000000000000000000000000000000 P 67fee47a4f5b4b02afdb9fa68b746571 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 67fee47a4f5b4b02afdb9fa68b746571, State: Initialized, Role: FOLLOWER
I20260806 03:51:02.282605 12279 external_mini_cluster.cc:1428] Setting key 4e75aea007c583f42f10e148fa5304f3
I20260806 03:51:02.282676 8738 consensus_queue.cc:260] T 00000000000000000000000000000000 P 67fee47a4f5b4b02afdb9fa68b746571 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "67fee47a4f5b4b02afdb9fa68b746571" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34629 } }
I20260806 03:51:02.282778 8738 raft_consensus.cc:388] T 00000000000000000000000000000000 P 67fee47a4f5b4b02afdb9fa68b746571 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 03:51:02.282848 8738 raft_consensus.cc:482] T 00000000000000000000000000000000 P 67fee47a4f5b4b02afdb9fa68b746571 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 03:51:02.282900 8738 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 67fee47a4f5b4b02afdb9fa68b746571 [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:51:02.284003 8738 raft_consensus.cc:504] T 00000000000000000000000000000000 P 67fee47a4f5b4b02afdb9fa68b746571 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "67fee47a4f5b4b02afdb9fa68b746571" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34629 } }
I20260806 03:51:02.284152 8738 leader_election.cc:302] T 00000000000000000000000000000000 P 67fee47a4f5b4b02afdb9fa68b746571 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 67fee47a4f5b4b02afdb9fa68b746571; no voters:
I20260806 03:51:02.284338 8738 leader_election.cc:288] T 00000000000000000000000000000000 P 67fee47a4f5b4b02afdb9fa68b746571 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 03:51:02.284483 8744 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 67fee47a4f5b4b02afdb9fa68b746571 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 03:51:02.284654 8738 sys_catalog.cc:564] T 00000000000000000000000000000000 P 67fee47a4f5b4b02afdb9fa68b746571 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 03:51:02.284687 8744 raft_consensus.cc:686] T 00000000000000000000000000000000 P 67fee47a4f5b4b02afdb9fa68b746571 [term 1 LEADER]: Becoming Leader. State: Replica: 67fee47a4f5b4b02afdb9fa68b746571, State: Running, Role: LEADER
I20260806 03:51:02.284888 8744 consensus_queue.cc:237] T 00000000000000000000000000000000 P 67fee47a4f5b4b02afdb9fa68b746571 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "67fee47a4f5b4b02afdb9fa68b746571" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34629 } }
I20260806 03:51:02.285184 8743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:02.283223 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55125 (local address 127.0.0.1:34629)
0806 03:51:02.283429 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0806 03:51:02.283441 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 03:51:02.283484 (+ 43us) server_negotiation.cc:407] Connection header received
0806 03:51:02.283628 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:51:02.283634 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:51:02.283660 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:51:02.283832 (+ 172us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:51:02.284636 (+ 804us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:51:02.284640 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:51:02.284641 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 03:51:02.284645 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:51:02.284705 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 03:51:02.284742 (+ 37us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:51:02.284785 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0806 03:51:02.284924 (+ 139us) server_negotiation.cc:299] Negotiation successful
0806 03:51:02.284968 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":101,"threads_started":1}
I20260806 03:51:02.285658 8745 sys_catalog.cc:455] T 00000000000000000000000000000000 P 67fee47a4f5b4b02afdb9fa68b746571 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "67fee47a4f5b4b02afdb9fa68b746571" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "67fee47a4f5b4b02afdb9fa68b746571" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34629 } } }
I20260806 03:51:02.285768 8745 sys_catalog.cc:458] T 00000000000000000000000000000000 P 67fee47a4f5b4b02afdb9fa68b746571 [sys.catalog]: This master's current role is: LEADER
I20260806 03:51:02.286005 8746 sys_catalog.cc:455] T 00000000000000000000000000000000 P 67fee47a4f5b4b02afdb9fa68b746571 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 67fee47a4f5b4b02afdb9fa68b746571. Latest consensus state: current_term: 1 leader_uuid: "67fee47a4f5b4b02afdb9fa68b746571" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "67fee47a4f5b4b02afdb9fa68b746571" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34629 } } }
I20260806 03:51:02.286072 8756 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 03:51:02.286145 8746 sys_catalog.cc:458] T 00000000000000000000000000000000 P 67fee47a4f5b4b02afdb9fa68b746571 [sys.catalog]: This master's current role is: LEADER
I20260806 03:51:02.286599 12279 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 03:51:02.287070 8756 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 03:51:02.289003 8756 catalog_manager.cc:1347] Generated new cluster ID: 753a474ae7b947b3b9b122dfa2f7724e
I20260806 03:51:02.289064 8756 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 03:51:02.291258 8743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:02.288359 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49907 (local address 127.0.0.1:34629)
0806 03:51:02.288474 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0806 03:51:02.288475 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 03:51:02.290141 (+ 1666us) server_negotiation.cc:407] Connection header received
0806 03:51:02.290177 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:51:02.290177 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:51:02.290184 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:51:02.290265 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:51:02.290993 (+ 728us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:51:02.290994 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:51:02.290995 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 03:51:02.290999 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:51:02.291018 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 03:51:02.291022 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:51:02.291096 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0806 03:51:02.291152 (+ 56us) server_negotiation.cc:299] Negotiation successful
0806 03:51:02.291166 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99}
I20260806 03:51:02.298839 8756 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 03:51:02.299055 8756 catalog_manager.cc:1504] Loading token signing keys...
I20260806 03:51:02.303818 8756 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 67fee47a4f5b4b02afdb9fa68b746571: Generated new TSK 0
I20260806 03:51:02.304044 8756 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 03:51:02.311591 12279 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskToVWRe/build/release/bin/kudu with pid 8650
2026-08-06T03:51:02Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1631 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785988004637995-12279-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 03:51:02 dist-test-slave-6ntk krb5kdc[8781](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 03:51:02 dist-test-slave-6ntk krb5kdc[8781](info): set up 2 sockets
Aug 06 03:51:02 dist-test-slave-6ntk krb5kdc[8781](info): commencing operation
krb5kdc: starting...
W20260806 03:51:03.594477 12279 mini_kdc.cc:121] Time spent starting KDC: real 1.251s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785988004637995-12279-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785988004637995-12279-0/krb5kdc/test-user.keytab.
Aug 06 03:51:03 dist-test-slave-6ntk krb5kdc[8781](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988263, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T03:51:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T03:51:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785988004637995-12279-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785988004637995-12279-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 03:51:03.727090 12279 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskToVWRe/build/release/bin/kudu
/tmp/dist-test-taskToVWRe/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785988004637995-12279-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785988004637995-12279-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785988004637995-12279-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785988004637995-12279-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46715
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785988004637995-12279-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46501
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46715
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785988004637995-12279-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785988004637995-12279-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785988004637995-12279-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:51:03.812775 8797 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:51:03.813032 8797 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:51:03.813076 8797 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:51:03.814324 8797 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 03:51:03.814397 8797 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:51:03.814424 8797 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:51:03.814457 8797 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 03:51:03.814482 8797 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 03:51:03.815999 8797 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46501
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785988004637995-12279-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785988004637995-12279-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46715
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785988004637995-12279-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46715
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785988004637995-12279-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8797
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785988004637995-12279-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:51:03.816289 8797 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:51:03.816540 8797 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:51:03.821664 8806 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:51:03.821653 8813 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:51:03.821650 8807 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:51:03.821979 8797 server_base.cc:1034] running on GCE node
I20260806 03:51:03.822206 8797 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:51:03.822510 8797 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:51:03.823658 8797 hybrid_clock.cc:648] HybridClock initialized: now 1785988263823635 us; error 45 us; skew 500 ppm
Aug 06 03:51:03 dist-test-slave-6ntk krb5kdc[8781](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988263, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:51:03.826351 8797 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 03:51:03.826889 8797 webserver.cc:460] Webserver started at http://127.0.0.1:43797/ using document root <none> and password file <none>
I20260806 03:51:03.827158 8797 fs_manager.cc:359] Metadata directory not provided
I20260806 03:51:03.827231 8797 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:51:03.827384 8797 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:51:03.828466 8797 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785988004637995-12279-0/minicluster-data/master-0/data/instance:
uuid: "2872b10d73ea4341a2a6ab4ccc7718c0"
format_stamp: "Formatted at 2026-08-06 03:51:03 on dist-test-slave-6ntk"
server_key: "b193548c5ae5e49ccec2062a199c98dd"
server_key_iv: "bb4f20a4c2d32e14333abf7f0c18c043"
server_key_version: "encryptionkey@0"
I20260806 03:51:03.828832 8797 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785988004637995-12279-0/minicluster-data/master-0/wal/instance:
uuid: "2872b10d73ea4341a2a6ab4ccc7718c0"
format_stamp: "Formatted at 2026-08-06 03:51:03 on dist-test-slave-6ntk"
server_key: "b193548c5ae5e49ccec2062a199c98dd"
server_key_iv: "bb4f20a4c2d32e14333abf7f0c18c043"
server_key_version: "encryptionkey@0"
I20260806 03:51:03.830365 8797 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 03:51:03.831164 8825 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:51:03.831414 8797 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 03:51:03.831512 8797 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785988004637995-12279-0/minicluster-data/master-0/data,/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785988004637995-12279-0/minicluster-data/master-0/wal
uuid: "2872b10d73ea4341a2a6ab4ccc7718c0"
format_stamp: "Formatted at 2026-08-06 03:51:03 on dist-test-slave-6ntk"
server_key: "b193548c5ae5e49ccec2062a199c98dd"
server_key_iv: "bb4f20a4c2d32e14333abf7f0c18c043"
server_key_version: "encryptionkey@0"
I20260806 03:51:03.831574 8797 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785988004637995-12279-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785988004637995-12279-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785988004637995-12279-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:51:03.846946 8797 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:51:03.849134 8797 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:51:03.849289 8797 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:51:03.853511 8797 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46715
I20260806 03:51:03.853549 8893 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46715 every 8 connection(s)
I20260806 03:51:03.853962 8797 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785988004637995-12279-0/minicluster-data/master-0/data/info.pb
I20260806 03:51:03.854925 8894 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:51:03.857357 8894 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2872b10d73ea4341a2a6ab4ccc7718c0: Bootstrap starting.
I20260806 03:51:03.857992 8894 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2872b10d73ea4341a2a6ab4ccc7718c0: Neither blocks nor log segments found. Creating new log.
I20260806 03:51:03.858296 8894 log.cc:824] T 00000000000000000000000000000000 P 2872b10d73ea4341a2a6ab4ccc7718c0: Log is configured to *not* fsync() on all Append() calls
I20260806 03:51:03.859053 8894 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2872b10d73ea4341a2a6ab4ccc7718c0: No bootstrap required, opened a new log
I20260806 03:51:03.860707 8894 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2872b10d73ea4341a2a6ab4ccc7718c0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2872b10d73ea4341a2a6ab4ccc7718c0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46715 } }
I20260806 03:51:03.860845 8894 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2872b10d73ea4341a2a6ab4ccc7718c0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:51:03.860868 8894 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2872b10d73ea4341a2a6ab4ccc7718c0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2872b10d73ea4341a2a6ab4ccc7718c0, State: Initialized, Role: FOLLOWER
I20260806 03:51:03.861042 8894 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2872b10d73ea4341a2a6ab4ccc7718c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2872b10d73ea4341a2a6ab4ccc7718c0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46715 } }
I20260806 03:51:03.861106 8894 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2872b10d73ea4341a2a6ab4ccc7718c0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 03:51:03.861127 8894 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2872b10d73ea4341a2a6ab4ccc7718c0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 03:51:03.861193 8894 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2872b10d73ea4341a2a6ab4ccc7718c0 [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:51:03.861752 8894 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2872b10d73ea4341a2a6ab4ccc7718c0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2872b10d73ea4341a2a6ab4ccc7718c0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46715 } }
I20260806 03:51:03.861869 8894 leader_election.cc:302] T 00000000000000000000000000000000 P 2872b10d73ea4341a2a6ab4ccc7718c0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2872b10d73ea4341a2a6ab4ccc7718c0; no voters:
I20260806 03:51:03.862135 8894 leader_election.cc:288] T 00000000000000000000000000000000 P 2872b10d73ea4341a2a6ab4ccc7718c0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 03:51:03.862175 8898 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2872b10d73ea4341a2a6ab4ccc7718c0 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 03:51:03.862396 8898 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2872b10d73ea4341a2a6ab4ccc7718c0 [term 1 LEADER]: Becoming Leader. State: Replica: 2872b10d73ea4341a2a6ab4ccc7718c0, State: Running, Role: LEADER
I20260806 03:51:03.862411 8894 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2872b10d73ea4341a2a6ab4ccc7718c0 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 03:51:03.862512 8898 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2872b10d73ea4341a2a6ab4ccc7718c0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2872b10d73ea4341a2a6ab4ccc7718c0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46715 } }
I20260806 03:51:03.862994 12279 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskToVWRe/build/release/bin/kudu as pid 8797
I20260806 03:51:03.863127 12279 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785988004637995-12279-0/minicluster-data/master-0/wal/instance
I20260806 03:51:03.863087 8899 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2872b10d73ea4341a2a6ab4ccc7718c0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2872b10d73ea4341a2a6ab4ccc7718c0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2872b10d73ea4341a2a6ab4ccc7718c0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46715 } } }
I20260806 03:51:03.863168 8899 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2872b10d73ea4341a2a6ab4ccc7718c0 [sys.catalog]: This master's current role is: LEADER
I20260806 03:51:03.863319 12279 external_mini_cluster.cc:1428] Setting key 9bb97ea670cfceb6e4e82c0033b6b2f7
I20260806 03:51:03.863538 8900 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2872b10d73ea4341a2a6ab4ccc7718c0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2872b10d73ea4341a2a6ab4ccc7718c0. Latest consensus state: current_term: 1 leader_uuid: "2872b10d73ea4341a2a6ab4ccc7718c0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2872b10d73ea4341a2a6ab4ccc7718c0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46715 } } }
I20260806 03:51:03.863669 8913 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 03:51:03.863682 8900 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2872b10d73ea4341a2a6ab4ccc7718c0 [sys.catalog]: This master's current role is: LEADER
I20260806 03:51:03.864418 8913 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 03:51:03.866976 8913 catalog_manager.cc:1347] Generated new cluster ID: 11a6b0f1aed44f9ab05888034e9ecde4
I20260806 03:51:03.867048 8913 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 03:51:03 dist-test-slave-6ntk krb5kdc[8781](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988263, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260806 03:51:03.875655 8928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:03.864339 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57465 (local address 127.0.0.1:46715)
0806 03:51:03.864635 (+ 296us) server_negotiation.cc:206] Beginning negotiation
0806 03:51:03.864646 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 03:51:03.864681 (+ 35us) server_negotiation.cc:407] Connection header received
0806 03:51:03.864796 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:51:03.864803 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:51:03.864859 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:51:03.865013 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:51:03.865826 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:03.866861 (+ 1035us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:03.867968 (+ 1107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:03.868592 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:03.868703 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:03.871667 (+ 2964us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:51:03.871672 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:51:03.871673 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:51:03.871687 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:51:03.873909 (+ 2222us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:51:03.874320 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:51:03.874324 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:51:03.874324 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:51:03.874456 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:51:03.874664 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:51:03.874665 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:51:03.874665 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:51:03.875002 (+ 337us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:51:03.875174 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0806 03:51:03.875250 (+ 76us) server_negotiation.cc:299] Negotiation successful
0806 03:51:03.875415 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":244,"thread_start_us":94,"threads_started":1}
I20260806 03:51:03.876633 12279 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 03:51:03.883792 8913 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 03:51:03.884266 8913 catalog_manager.cc:1504] Loading token signing keys...
I20260806 03:51:03.885545 8928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:03.878076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46989 (local address 127.0.0.1:46715)
0806 03:51:03.878168 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0806 03:51:03.878170 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:51:03.878217 (+ 47us) server_negotiation.cc:407] Connection header received
0806 03:51:03.878289 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:51:03.878290 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:51:03.878300 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:51:03.878372 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:51:03.878960 (+ 588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:03.879769 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:03.880767 (+ 998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:03.881093 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:03.881166 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:03.882556 (+ 1390us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:51:03.882558 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:51:03.882558 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:51:03.882565 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:51:03.884289 (+ 1724us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:51:03.884752 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:51:03.884753 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:51:03.884753 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:51:03.884820 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:51:03.885032 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:51:03.885033 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:51:03.885033 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:51:03.885142 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:51:03.885228 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0806 03:51:03.885372 (+ 144us) server_negotiation.cc:299] Negotiation successful
0806 03:51:03.885460 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260806 03:51:03.891141 8913 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2872b10d73ea4341a2a6ab4ccc7718c0: Generated new TSK 0
I20260806 03:51:03.891460 8913 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 03:51:03.895144 12279 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskToVWRe/build/release/bin/kudu with pid 8797
2026-08-06T03:51:03Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1576 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785988004637995-12279-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 03:51:03 dist-test-slave-6ntk krb5kdc[8938](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 03:51:03 dist-test-slave-6ntk krb5kdc[8938](info): set up 2 sockets
Aug 06 03:51:03 dist-test-slave-6ntk krb5kdc[8938](info): commencing operation
krb5kdc: starting...
W20260806 03:51:05.135847 12279 mini_kdc.cc:121] Time spent starting KDC: real 1.216s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785988004637995-12279-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785988004637995-12279-0/krb5kdc/test-user.keytab.
Aug 06 03:51:05 dist-test-slave-6ntk krb5kdc[8938](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988265, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T03:51:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T03:51:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785988004637995-12279-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785988004637995-12279-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 03:51:05.277596 12279 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskToVWRe/build/release/bin/kudu
/tmp/dist-test-taskToVWRe/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785988004637995-12279-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785988004637995-12279-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785988004637995-12279-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785988004637995-12279-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45733
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785988004637995-12279-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37793
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45733
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785988004637995-12279-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785988004637995-12279-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785988004637995-12279-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:51:05.365170 8953 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:51:05.365474 8953 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:51:05.365523 8953 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:51:05.366889 8953 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 03:51:05.366986 8953 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:51:05.367018 8953 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:51:05.367050 8953 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 03:51:05.367076 8953 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 03:51:05.368780 8953 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37793
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785988004637995-12279-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785988004637995-12279-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45733
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785988004637995-12279-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45733
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785988004637995-12279-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8953
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785988004637995-12279-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:51:05.369099 8953 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:51:05.369380 8953 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:51:05.374711 8967 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:51:05.374732 8961 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:51:05.374719 8958 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:51:05.375015 8953 server_base.cc:1034] running on GCE node
I20260806 03:51:05.375340 8953 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:51:05.375716 8953 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:51:05.376946 8953 hybrid_clock.cc:648] HybridClock initialized: now 1785988265376919 us; error 37 us; skew 500 ppm
Aug 06 03:51:05 dist-test-slave-6ntk krb5kdc[8938](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988265, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:51:05.384760 8953 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 03:51:05.385308 8953 webserver.cc:460] Webserver started at http://127.0.0.1:43095/ using document root <none> and password file <none>
I20260806 03:51:05.385550 8953 fs_manager.cc:359] Metadata directory not provided
I20260806 03:51:05.385622 8953 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:51:05.385789 8953 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:51:05.387089 8953 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785988004637995-12279-0/minicluster-data/master-0/data/instance:
uuid: "7da343152abd4c6aa344f29e34cb536c"
format_stamp: "Formatted at 2026-08-06 03:51:05 on dist-test-slave-6ntk"
server_key: "82f4027b1686153762af677791594dbf"
server_key_iv: "05a7e8623a92303556906f5beff8b604"
server_key_version: "encryptionkey@0"
I20260806 03:51:05.387483 8953 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785988004637995-12279-0/minicluster-data/master-0/wal/instance:
uuid: "7da343152abd4c6aa344f29e34cb536c"
format_stamp: "Formatted at 2026-08-06 03:51:05 on dist-test-slave-6ntk"
server_key: "82f4027b1686153762af677791594dbf"
server_key_iv: "05a7e8623a92303556906f5beff8b604"
server_key_version: "encryptionkey@0"
I20260806 03:51:05.388948 8953 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 03:51:05.389719 8975 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:51:05.389957 8953 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 03:51:05.390067 8953 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785988004637995-12279-0/minicluster-data/master-0/data,/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785988004637995-12279-0/minicluster-data/master-0/wal
uuid: "7da343152abd4c6aa344f29e34cb536c"
format_stamp: "Formatted at 2026-08-06 03:51:05 on dist-test-slave-6ntk"
server_key: "82f4027b1686153762af677791594dbf"
server_key_iv: "05a7e8623a92303556906f5beff8b604"
server_key_version: "encryptionkey@0"
I20260806 03:51:05.390159 8953 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785988004637995-12279-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785988004637995-12279-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785988004637995-12279-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:51:05.413908 8953 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:51:05.416332 8953 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:51:05.416535 8953 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:51:05.421131 8953 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45733
I20260806 03:51:05.421185 9042 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45733 every 8 connection(s)
I20260806 03:51:05.421597 8953 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785988004637995-12279-0/minicluster-data/master-0/data/info.pb
I20260806 03:51:05.422598 9043 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:51:05.424569 12279 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskToVWRe/build/release/bin/kudu as pid 8953
I20260806 03:51:05.424702 12279 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785988004637995-12279-0/minicluster-data/master-0/wal/instance
I20260806 03:51:05.424921 12279 external_mini_cluster.cc:1428] Setting key a8de28513cac3f1d48854d5dbb736795
I20260806 03:51:05.425470 9043 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7da343152abd4c6aa344f29e34cb536c: Bootstrap starting.
I20260806 03:51:05.426299 9043 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7da343152abd4c6aa344f29e34cb536c: Neither blocks nor log segments found. Creating new log.
I20260806 03:51:05.426649 9043 log.cc:824] T 00000000000000000000000000000000 P 7da343152abd4c6aa344f29e34cb536c: Log is configured to *not* fsync() on all Append() calls
I20260806 03:51:05.427417 9043 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7da343152abd4c6aa344f29e34cb536c: No bootstrap required, opened a new log
I20260806 03:51:05.429407 9043 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7da343152abd4c6aa344f29e34cb536c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7da343152abd4c6aa344f29e34cb536c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45733 } }
I20260806 03:51:05.429589 9043 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7da343152abd4c6aa344f29e34cb536c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:51:05.429636 9043 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7da343152abd4c6aa344f29e34cb536c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7da343152abd4c6aa344f29e34cb536c, State: Initialized, Role: FOLLOWER
I20260806 03:51:05.429791 9043 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7da343152abd4c6aa344f29e34cb536c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7da343152abd4c6aa344f29e34cb536c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45733 } }
I20260806 03:51:05.429929 9043 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7da343152abd4c6aa344f29e34cb536c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 03:51:05.429973 9043 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7da343152abd4c6aa344f29e34cb536c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 03:51:05.430023 9043 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7da343152abd4c6aa344f29e34cb536c [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:51:05.430686 9043 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7da343152abd4c6aa344f29e34cb536c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7da343152abd4c6aa344f29e34cb536c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45733 } }
I20260806 03:51:05.430790 9043 leader_election.cc:302] T 00000000000000000000000000000000 P 7da343152abd4c6aa344f29e34cb536c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7da343152abd4c6aa344f29e34cb536c; no voters:
I20260806 03:51:05.431003 9043 leader_election.cc:288] T 00000000000000000000000000000000 P 7da343152abd4c6aa344f29e34cb536c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 03:51:05.431143 9050 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7da343152abd4c6aa344f29e34cb536c [term 1 FOLLOWER]: Leader election won for term 1
I20260806 03:51:05.431250 9047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:05.425529 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34185 (local address 127.0.0.1:45733)
0806 03:51:05.425726 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0806 03:51:05.425736 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 03:51:05.425770 (+ 34us) server_negotiation.cc:407] Connection header received
0806 03:51:05.426492 (+ 722us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:51:05.426505 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:51:05.426578 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:51:05.426740 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:51:05.427036 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:05.428077 (+ 1041us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:05.428924 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:05.429396 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:05.429551 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:05.430487 (+ 936us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:51:05.430492 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:51:05.430492 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 03:51:05.430683 (+ 191us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:51:05.430723 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 03:51:05.430736 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:51:05.430829 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0806 03:51:05.430938 (+ 109us) server_negotiation.cc:299] Negotiation successful
0806 03:51:05.431010 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":92,"threads_started":1}
I20260806 03:51:05.431277 9050 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7da343152abd4c6aa344f29e34cb536c [term 1 LEADER]: Becoming Leader. State: Replica: 7da343152abd4c6aa344f29e34cb536c, State: Running, Role: LEADER
I20260806 03:51:05.431309 9043 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7da343152abd4c6aa344f29e34cb536c [sys.catalog]: configured and running, proceeding with master startup.
I20260806 03:51:05.431383 9050 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7da343152abd4c6aa344f29e34cb536c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7da343152abd4c6aa344f29e34cb536c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45733 } }
I20260806 03:51:05.431846 9052 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7da343152abd4c6aa344f29e34cb536c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7da343152abd4c6aa344f29e34cb536c. Latest consensus state: current_term: 1 leader_uuid: "7da343152abd4c6aa344f29e34cb536c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7da343152abd4c6aa344f29e34cb536c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45733 } } }
I20260806 03:51:05.431959 9052 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7da343152abd4c6aa344f29e34cb536c [sys.catalog]: This master's current role is: LEADER
I20260806 03:51:05.432220 9055 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 03:51:05.432741 12279 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 03:51:05.432760 9055 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 03:51:05.431838 9051 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7da343152abd4c6aa344f29e34cb536c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7da343152abd4c6aa344f29e34cb536c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7da343152abd4c6aa344f29e34cb536c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45733 } } }
I20260806 03:51:05.434243 9051 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7da343152abd4c6aa344f29e34cb536c [sys.catalog]: This master's current role is: LEADER
I20260806 03:51:05.434857 9055 catalog_manager.cc:1347] Generated new cluster ID: 0a8ec8ade47c4b3988f5b878947d8d19
I20260806 03:51:05.434916 9055 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 03:51:05.438206 9047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:05.434382 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50475 (local address 127.0.0.1:45733)
0806 03:51:05.434420 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0806 03:51:05.434423 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 03:51:05.434621 (+ 198us) server_negotiation.cc:407] Connection header received
0806 03:51:05.434736 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:51:05.434737 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:51:05.434748 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:51:05.434777 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:51:05.435016 (+ 239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:05.435753 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:05.436788 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:05.437084 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:05.437120 (+ 36us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:05.437960 (+ 840us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:51:05.437961 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:51:05.437961 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 03:51:05.437966 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:51:05.437981 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 03:51:05.437985 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:51:05.438010 (+ 25us) server_negotiation.cc:1035] Waiting for connection context
0806 03:51:05.438104 (+ 94us) server_negotiation.cc:299] Negotiation successful
0806 03:51:05.438129 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260806 03:51:05.439821 9055 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 03:51:05.440387 9055 catalog_manager.cc:1504] Loading token signing keys...
I20260806 03:51:05.445832 9055 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7da343152abd4c6aa344f29e34cb536c: Generated new TSK 0
I20260806 03:51:05.446071 9055 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 03:51:05.449587 12279 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskToVWRe/build/release/bin/kudu with pid 8953
2026-08-06T03:51:05Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1564 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785988004637995-12279-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 03:51:05 dist-test-slave-6ntk krb5kdc[9089](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 03:51:05 dist-test-slave-6ntk krb5kdc[9089](info): set up 2 sockets
Aug 06 03:51:05 dist-test-slave-6ntk krb5kdc[9089](info): commencing operation
krb5kdc: starting...
W20260806 03:51:06.692278 12279 mini_kdc.cc:121] Time spent starting KDC: real 1.208s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785988004637995-12279-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785988004637995-12279-0/krb5kdc/test-user.keytab.
Aug 06 03:51:06 dist-test-slave-6ntk krb5kdc[9089](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988266, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T03:51:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T03:51:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785988004637995-12279-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785988004637995-12279-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 03:51:06.825711 12279 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskToVWRe/build/release/bin/kudu
/tmp/dist-test-taskToVWRe/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785988004637995-12279-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785988004637995-12279-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785988004637995-12279-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785988004637995-12279-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43187
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785988004637995-12279-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41383
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43187
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785988004637995-12279-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785988004637995-12279-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785988004637995-12279-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:51:06.911556 9105 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:51:06.911823 9105 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:51:06.911864 9105 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:51:06.913034 9105 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 03:51:06.913107 9105 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:51:06.913131 9105 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:51:06.913153 9105 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 03:51:06.913182 9105 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 03:51:06.915304 9105 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41383
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785988004637995-12279-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785988004637995-12279-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43187
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785988004637995-12279-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43187
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785988004637995-12279-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9105
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785988004637995-12279-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:51:06.915576 9105 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:51:06.915839 9105 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:51:06.921088 9116 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:51:06.921097 9113 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:51:06.921185 9105 server_base.cc:1034] running on GCE node
W20260806 03:51:06.921111 9114 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:51:06.921586 9105 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:51:06.921959 9105 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:51:06.923134 9105 hybrid_clock.cc:648] HybridClock initialized: now 1785988266923117 us; error 51 us; skew 500 ppm
Aug 06 03:51:06 dist-test-slave-6ntk krb5kdc[9089](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988266, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:51:06.925791 9105 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 03:51:06.926352 9105 webserver.cc:460] Webserver started at http://127.0.0.1:46467/ using document root <none> and password file <none>
I20260806 03:51:06.926620 9105 fs_manager.cc:359] Metadata directory not provided
I20260806 03:51:06.926668 9105 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:51:06.926802 9105 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:51:06.927889 9105 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785988004637995-12279-0/minicluster-data/master-0/data/instance:
uuid: "a0b3bb3417b2487c88b315278fa57f38"
format_stamp: "Formatted at 2026-08-06 03:51:06 on dist-test-slave-6ntk"
server_key: "204188f008f1b000e5e422a907a2fc35"
server_key_iv: "7fec64993154b37a5612acb17fdba423"
server_key_version: "encryptionkey@0"
I20260806 03:51:06.928328 9105 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785988004637995-12279-0/minicluster-data/master-0/wal/instance:
uuid: "a0b3bb3417b2487c88b315278fa57f38"
format_stamp: "Formatted at 2026-08-06 03:51:06 on dist-test-slave-6ntk"
server_key: "204188f008f1b000e5e422a907a2fc35"
server_key_iv: "7fec64993154b37a5612acb17fdba423"
server_key_version: "encryptionkey@0"
I20260806 03:51:06.929716 9105 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 03:51:06.930569 9126 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:51:06.930768 9105 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 03:51:06.930869 9105 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785988004637995-12279-0/minicluster-data/master-0/data,/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785988004637995-12279-0/minicluster-data/master-0/wal
uuid: "a0b3bb3417b2487c88b315278fa57f38"
format_stamp: "Formatted at 2026-08-06 03:51:06 on dist-test-slave-6ntk"
server_key: "204188f008f1b000e5e422a907a2fc35"
server_key_iv: "7fec64993154b37a5612acb17fdba423"
server_key_version: "encryptionkey@0"
I20260806 03:51:06.930949 9105 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785988004637995-12279-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785988004637995-12279-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785988004637995-12279-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:51:06.934378 9105 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:51:06.936740 9105 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:51:06.936894 9105 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:51:06.941222 9105 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43187
I20260806 03:51:06.941284 9193 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43187 every 8 connection(s)
I20260806 03:51:06.941615 9105 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785988004637995-12279-0/minicluster-data/master-0/data/info.pb
I20260806 03:51:06.941938 12279 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskToVWRe/build/release/bin/kudu as pid 9105
I20260806 03:51:06.942095 12279 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785988004637995-12279-0/minicluster-data/master-0/wal/instance
I20260806 03:51:06.942312 12279 external_mini_cluster.cc:1428] Setting key 0a6ba2da22db9a2acfce08832d88d61f
I20260806 03:51:06.942675 9194 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:51:06.945209 9199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:06.942965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34243 (local address 127.0.0.1:43187)
0806 03:51:06.943207 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0806 03:51:06.943217 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 03:51:06.943256 (+ 39us) server_negotiation.cc:407] Connection header received
0806 03:51:06.943900 (+ 644us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:51:06.943909 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:51:06.943945 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:51:06.944100 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:51:06.944759 (+ 659us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:51:06.944763 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:51:06.944764 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 03:51:06.944768 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:51:06.944816 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 03:51:06.944832 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:51:06.944907 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0806 03:51:06.944964 (+ 57us) server_negotiation.cc:299] Negotiation successful
0806 03:51:06.945004 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":178,"thread_start_us":118,"threads_started":1}
I20260806 03:51:06.946338 12279 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 03:51:06.946478 9194 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a0b3bb3417b2487c88b315278fa57f38: Bootstrap starting.
I20260806 03:51:06.947257 9194 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a0b3bb3417b2487c88b315278fa57f38: Neither blocks nor log segments found. Creating new log.
I20260806 03:51:06.947602 9194 log.cc:824] T 00000000000000000000000000000000 P a0b3bb3417b2487c88b315278fa57f38: Log is configured to *not* fsync() on all Append() calls
I20260806 03:51:06.948482 9194 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a0b3bb3417b2487c88b315278fa57f38: No bootstrap required, opened a new log
I20260806 03:51:06.949317 9199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:06.948054 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37879 (local address 127.0.0.1:43187)
0806 03:51:06.948089 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0806 03:51:06.948091 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:51:06.948238 (+ 147us) server_negotiation.cc:407] Connection header received
0806 03:51:06.948360 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:51:06.948361 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:51:06.948367 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:51:06.948416 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:51:06.948975 (+ 559us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:51:06.948976 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:51:06.948976 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 03:51:06.948980 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:51:06.948994 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 03:51:06.948997 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:51:06.949042 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0806 03:51:06.949123 (+ 81us) server_negotiation.cc:299] Negotiation successful
0806 03:51:06.949134 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260806 03:51:06.950897 9194 raft_consensus.cc:348] T 00000000000000000000000000000000 P a0b3bb3417b2487c88b315278fa57f38 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0b3bb3417b2487c88b315278fa57f38" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43187 } }
I20260806 03:51:06.951045 9194 raft_consensus.cc:374] T 00000000000000000000000000000000 P a0b3bb3417b2487c88b315278fa57f38 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:51:06.951069 9194 raft_consensus.cc:729] T 00000000000000000000000000000000 P a0b3bb3417b2487c88b315278fa57f38 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0b3bb3417b2487c88b315278fa57f38, State: Initialized, Role: FOLLOWER
I20260806 03:51:06.951217 9194 consensus_queue.cc:260] T 00000000000000000000000000000000 P a0b3bb3417b2487c88b315278fa57f38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0b3bb3417b2487c88b315278fa57f38" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43187 } }
I20260806 03:51:06.951306 9194 raft_consensus.cc:388] T 00000000000000000000000000000000 P a0b3bb3417b2487c88b315278fa57f38 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 03:51:06.951354 9194 raft_consensus.cc:482] T 00000000000000000000000000000000 P a0b3bb3417b2487c88b315278fa57f38 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 03:51:06.951404 9194 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a0b3bb3417b2487c88b315278fa57f38 [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:51:06.951967 9194 raft_consensus.cc:504] T 00000000000000000000000000000000 P a0b3bb3417b2487c88b315278fa57f38 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0b3bb3417b2487c88b315278fa57f38" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43187 } }
I20260806 03:51:06.952085 9194 leader_election.cc:302] T 00000000000000000000000000000000 P a0b3bb3417b2487c88b315278fa57f38 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a0b3bb3417b2487c88b315278fa57f38; no voters:
I20260806 03:51:06.952257 9194 leader_election.cc:288] T 00000000000000000000000000000000 P a0b3bb3417b2487c88b315278fa57f38 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 03:51:06.952368 9206 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a0b3bb3417b2487c88b315278fa57f38 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 03:51:06.952521 9194 sys_catalog.cc:564] T 00000000000000000000000000000000 P a0b3bb3417b2487c88b315278fa57f38 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 03:51:06.952530 9206 raft_consensus.cc:686] T 00000000000000000000000000000000 P a0b3bb3417b2487c88b315278fa57f38 [term 1 LEADER]: Becoming Leader. State: Replica: a0b3bb3417b2487c88b315278fa57f38, State: Running, Role: LEADER
I20260806 03:51:06.952637 9206 consensus_queue.cc:237] T 00000000000000000000000000000000 P a0b3bb3417b2487c88b315278fa57f38 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0b3bb3417b2487c88b315278fa57f38" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43187 } }
I20260806 03:51:06.953007 9207 sys_catalog.cc:455] T 00000000000000000000000000000000 P a0b3bb3417b2487c88b315278fa57f38 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a0b3bb3417b2487c88b315278fa57f38" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0b3bb3417b2487c88b315278fa57f38" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43187 } } }
I20260806 03:51:06.953152 9207 sys_catalog.cc:458] T 00000000000000000000000000000000 P a0b3bb3417b2487c88b315278fa57f38 [sys.catalog]: This master's current role is: LEADER
I20260806 03:51:06.953011 9208 sys_catalog.cc:455] T 00000000000000000000000000000000 P a0b3bb3417b2487c88b315278fa57f38 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a0b3bb3417b2487c88b315278fa57f38. Latest consensus state: current_term: 1 leader_uuid: "a0b3bb3417b2487c88b315278fa57f38" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0b3bb3417b2487c88b315278fa57f38" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43187 } } }
I20260806 03:51:06.953336 9208 sys_catalog.cc:458] T 00000000000000000000000000000000 P a0b3bb3417b2487c88b315278fa57f38 [sys.catalog]: This master's current role is: LEADER
W20260806 03:51:06.954381 9225 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a0b3bb3417b2487c88b315278fa57f38: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 03:51:06.954450 9225 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 03:51:06.954510 9216 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 03:51:06.954651 9216 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 03:51:06.956204 9216 catalog_manager.cc:1347] Generated new cluster ID: 94eaf9b391494e9290e502518f956819
I20260806 03:51:06.956274 9216 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 03:51:06.968936 9216 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 03:51:06.969151 9216 catalog_manager.cc:1504] Loading token signing keys...
I20260806 03:51:06.980253 9216 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a0b3bb3417b2487c88b315278fa57f38: Generated new TSK 0
I20260806 03:51:06.980513 9216 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 03:51:06.984790 12279 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskToVWRe/build/release/bin/kudu with pid 9105
2026-08-06T03:51:06Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1527 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785988004637995-12279-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 03:51:07 dist-test-slave-6ntk krb5kdc[9230](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 03:51:07 dist-test-slave-6ntk krb5kdc[9230](info): set up 2 sockets
Aug 06 03:51:07 dist-test-slave-6ntk krb5kdc[9230](info): commencing operation
krb5kdc: starting...
W20260806 03:51:08.227358 12279 mini_kdc.cc:121] Time spent starting KDC: real 1.215s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785988004637995-12279-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785988004637995-12279-0/krb5kdc/test-user.keytab.
Aug 06 03:51:08 dist-test-slave-6ntk krb5kdc[9230](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988268, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T03:51:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T03:51:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785988004637995-12279-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785988004637995-12279-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 03:51:08.365517 12279 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskToVWRe/build/release/bin/kudu
/tmp/dist-test-taskToVWRe/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785988004637995-12279-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785988004637995-12279-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785988004637995-12279-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785988004637995-12279-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42933
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785988004637995-12279-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37891
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42933
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785988004637995-12279-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785988004637995-12279-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785988004637995-12279-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:51:08.451857 9254 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:51:08.452111 9254 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:51:08.452150 9254 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:51:08.453272 9254 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 03:51:08.453343 9254 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:51:08.453368 9254 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:51:08.453390 9254 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 03:51:08.453418 9254 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 03:51:08.454881 9254 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37891
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785988004637995-12279-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785988004637995-12279-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42933
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785988004637995-12279-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42933
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785988004637995-12279-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9254
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785988004637995-12279-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:51:08.455160 9254 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:51:08.455394 9254 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:51:08.460373 9261 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:51:08.460382 9265 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:51:08.460415 9262 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:51:08.460932 9254 server_base.cc:1034] running on GCE node
I20260806 03:51:08.461181 9254 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:51:08.461498 9254 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:51:08.462698 9254 hybrid_clock.cc:648] HybridClock initialized: now 1785988268462680 us; error 45 us; skew 500 ppm
Aug 06 03:51:08 dist-test-slave-6ntk krb5kdc[9230](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988268, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:51:08.465378 9254 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 03:51:08.465906 9254 webserver.cc:460] Webserver started at http://127.0.0.1:44927/ using document root <none> and password file <none>
I20260806 03:51:08.466125 9254 fs_manager.cc:359] Metadata directory not provided
I20260806 03:51:08.466166 9254 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:51:08.466267 9254 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:51:08.467298 9254 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785988004637995-12279-0/minicluster-data/master-0/data/instance:
uuid: "f8508a2135ac4bd3bba98e9f652d588f"
format_stamp: "Formatted at 2026-08-06 03:51:08 on dist-test-slave-6ntk"
server_key: "fff387fc1ec493bc2bab8cfe7765990e"
server_key_iv: "baa7e0214e018432a9299c5136b8d7b8"
server_key_version: "encryptionkey@0"
I20260806 03:51:08.467617 9254 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785988004637995-12279-0/minicluster-data/master-0/wal/instance:
uuid: "f8508a2135ac4bd3bba98e9f652d588f"
format_stamp: "Formatted at 2026-08-06 03:51:08 on dist-test-slave-6ntk"
server_key: "fff387fc1ec493bc2bab8cfe7765990e"
server_key_iv: "baa7e0214e018432a9299c5136b8d7b8"
server_key_version: "encryptionkey@0"
I20260806 03:51:08.468915 9254 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 03:51:08.469669 9275 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:51:08.469853 9254 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 03:51:08.470001 9254 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785988004637995-12279-0/minicluster-data/master-0/data,/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785988004637995-12279-0/minicluster-data/master-0/wal
uuid: "f8508a2135ac4bd3bba98e9f652d588f"
format_stamp: "Formatted at 2026-08-06 03:51:08 on dist-test-slave-6ntk"
server_key: "fff387fc1ec493bc2bab8cfe7765990e"
server_key_iv: "baa7e0214e018432a9299c5136b8d7b8"
server_key_version: "encryptionkey@0"
I20260806 03:51:08.470086 9254 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785988004637995-12279-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785988004637995-12279-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785988004637995-12279-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:51:08.486330 9254 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:51:08.488286 9254 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:51:08.488461 9254 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:51:08.492805 9254 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42933
I20260806 03:51:08.492841 9343 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42933 every 8 connection(s)
I20260806 03:51:08.493204 9254 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785988004637995-12279-0/minicluster-data/master-0/data/info.pb
I20260806 03:51:08.494210 9344 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:51:08.496527 9344 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f8508a2135ac4bd3bba98e9f652d588f: Bootstrap starting.
I20260806 03:51:08.497076 9344 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f8508a2135ac4bd3bba98e9f652d588f: Neither blocks nor log segments found. Creating new log.
I20260806 03:51:08.497342 9344 log.cc:824] T 00000000000000000000000000000000 P f8508a2135ac4bd3bba98e9f652d588f: Log is configured to *not* fsync() on all Append() calls
I20260806 03:51:08.498045 9344 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f8508a2135ac4bd3bba98e9f652d588f: No bootstrap required, opened a new log
I20260806 03:51:08.499646 9344 raft_consensus.cc:348] T 00000000000000000000000000000000 P f8508a2135ac4bd3bba98e9f652d588f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8508a2135ac4bd3bba98e9f652d588f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42933 } }
I20260806 03:51:08.499783 9344 raft_consensus.cc:374] T 00000000000000000000000000000000 P f8508a2135ac4bd3bba98e9f652d588f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:51:08.499806 9344 raft_consensus.cc:729] T 00000000000000000000000000000000 P f8508a2135ac4bd3bba98e9f652d588f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f8508a2135ac4bd3bba98e9f652d588f, State: Initialized, Role: FOLLOWER
I20260806 03:51:08.499950 9344 consensus_queue.cc:260] T 00000000000000000000000000000000 P f8508a2135ac4bd3bba98e9f652d588f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8508a2135ac4bd3bba98e9f652d588f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42933 } }
I20260806 03:51:08.500039 9344 raft_consensus.cc:388] T 00000000000000000000000000000000 P f8508a2135ac4bd3bba98e9f652d588f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 03:51:08.500062 9344 raft_consensus.cc:482] T 00000000000000000000000000000000 P f8508a2135ac4bd3bba98e9f652d588f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 03:51:08.500092 9344 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f8508a2135ac4bd3bba98e9f652d588f [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:51:08.500582 9344 raft_consensus.cc:504] T 00000000000000000000000000000000 P f8508a2135ac4bd3bba98e9f652d588f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8508a2135ac4bd3bba98e9f652d588f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42933 } }
I20260806 03:51:08.500666 9344 leader_election.cc:302] T 00000000000000000000000000000000 P f8508a2135ac4bd3bba98e9f652d588f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f8508a2135ac4bd3bba98e9f652d588f; no voters:
I20260806 03:51:08.500792 9344 leader_election.cc:288] T 00000000000000000000000000000000 P f8508a2135ac4bd3bba98e9f652d588f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 03:51:08.500880 9349 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f8508a2135ac4bd3bba98e9f652d588f [term 1 FOLLOWER]: Leader election won for term 1
I20260806 03:51:08.501066 9349 raft_consensus.cc:686] T 00000000000000000000000000000000 P f8508a2135ac4bd3bba98e9f652d588f [term 1 LEADER]: Becoming Leader. State: Replica: f8508a2135ac4bd3bba98e9f652d588f, State: Running, Role: LEADER
I20260806 03:51:08.501091 9344 sys_catalog.cc:564] T 00000000000000000000000000000000 P f8508a2135ac4bd3bba98e9f652d588f [sys.catalog]: configured and running, proceeding with master startup.
I20260806 03:51:08.501206 9349 consensus_queue.cc:237] T 00000000000000000000000000000000 P f8508a2135ac4bd3bba98e9f652d588f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8508a2135ac4bd3bba98e9f652d588f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42933 } }
I20260806 03:51:08.501706 12279 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskToVWRe/build/release/bin/kudu as pid 9254
I20260806 03:51:08.501775 9350 sys_catalog.cc:455] T 00000000000000000000000000000000 P f8508a2135ac4bd3bba98e9f652d588f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f8508a2135ac4bd3bba98e9f652d588f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8508a2135ac4bd3bba98e9f652d588f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42933 } } }
I20260806 03:51:08.501856 12279 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785988004637995-12279-0/minicluster-data/master-0/wal/instance
I20260806 03:51:08.501909 9350 sys_catalog.cc:458] T 00000000000000000000000000000000 P f8508a2135ac4bd3bba98e9f652d588f [sys.catalog]: This master's current role is: LEADER
I20260806 03:51:08.502101 12279 external_mini_cluster.cc:1428] Setting key d5d9add634eeb9960181a6d45d4fb324
I20260806 03:51:08.502296 9351 sys_catalog.cc:455] T 00000000000000000000000000000000 P f8508a2135ac4bd3bba98e9f652d588f [sys.catalog]: SysCatalogTable state changed. Reason: New leader f8508a2135ac4bd3bba98e9f652d588f. Latest consensus state: current_term: 1 leader_uuid: "f8508a2135ac4bd3bba98e9f652d588f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8508a2135ac4bd3bba98e9f652d588f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42933 } } }
I20260806 03:51:08.502411 9351 sys_catalog.cc:458] T 00000000000000000000000000000000 P f8508a2135ac4bd3bba98e9f652d588f [sys.catalog]: This master's current role is: LEADER
I20260806 03:51:08.502676 9358 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 03:51:08.503298 9358 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 03:51:08.505007 9358 catalog_manager.cc:1347] Generated new cluster ID: 8227a53f363b451a95655d7c5c21b456
I20260806 03:51:08.505074 9358 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 03:51:08 dist-test-slave-6ntk krb5kdc[9230](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988268, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260806 03:51:08.514619 9365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:08.502639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.193:51269 (local address 127.0.0.1:42933)
0806 03:51:08.502858 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0806 03:51:08.502868 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 03:51:08.502917 (+ 49us) server_negotiation.cc:407] Connection header received
0806 03:51:08.503041 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:51:08.503049 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:51:08.503110 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:51:08.503271 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:51:08.504073 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:08.505463 (+ 1390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:08.506671 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:08.507301 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:08.507412 (+ 111us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:08.510528 (+ 3116us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:51:08.510533 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:51:08.510533 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:51:08.510547 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:51:08.512677 (+ 2130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:51:08.513142 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:51:08.513145 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:51:08.513145 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:51:08.513273 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:51:08.513532 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:51:08.513533 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:51:08.513534 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:51:08.513912 (+ 378us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:51:08.514088 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0806 03:51:08.514236 (+ 148us) server_negotiation.cc:299] Negotiation successful
0806 03:51:08.514350 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":164,"thread_start_us":80,"threads_started":1}
I20260806 03:51:08.516053 12279 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 03:51:08.516074 9358 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 03:51:08.516572 9358 catalog_manager.cc:1504] Loading token signing keys...
I20260806 03:51:08.525414 9365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:08.517691 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.193:53635 (local address 127.0.0.1:42933)
0806 03:51:08.517731 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0806 03:51:08.517733 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:51:08.517936 (+ 203us) server_negotiation.cc:407] Connection header received
0806 03:51:08.517964 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:51:08.517965 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:51:08.517976 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:51:08.518041 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:51:08.518668 (+ 627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:08.519487 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:08.520524 (+ 1037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:08.520840 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:08.520914 (+ 74us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:08.522409 (+ 1495us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:51:08.522411 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:51:08.522411 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:51:08.522419 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:51:08.524091 (+ 1672us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:51:08.524523 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:51:08.524525 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:51:08.524526 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:51:08.524589 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:51:08.524827 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:51:08.524828 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:51:08.524828 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:51:08.524957 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:51:08.525061 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0806 03:51:08.525238 (+ 177us) server_negotiation.cc:299] Negotiation successful
0806 03:51:08.525313 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 03:51:08.525985 9358 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f8508a2135ac4bd3bba98e9f652d588f: Generated new TSK 0
I20260806 03:51:08.526211 9358 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 03:51:08.532861 12279 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskToVWRe/build/release/bin/kudu with pid 9254
W20260806 03:51:08.547642 9228 connection.cc:535] client connection to 127.0.0.1:42933 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:42933): Connection reset by peer (error 104)
2026-08-06T03:51:08Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1549 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785988004637995-12279-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 03:51:08 dist-test-slave-6ntk krb5kdc[9386](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 03:51:08 dist-test-slave-6ntk krb5kdc[9386](info): set up 2 sockets
Aug 06 03:51:08 dist-test-slave-6ntk krb5kdc[9386](info): commencing operation
krb5kdc: starting...
W20260806 03:51:09.745643 12279 mini_kdc.cc:121] Time spent starting KDC: real 1.184s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785988004637995-12279-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785988004637995-12279-0/krb5kdc/test-user.keytab.
Aug 06 03:51:09 dist-test-slave-6ntk krb5kdc[9386](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988269, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T03:51:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T03:51:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785988004637995-12279-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785988004637995-12279-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 03:51:09.882081 12279 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskToVWRe/build/release/bin/kudu
/tmp/dist-test-taskToVWRe/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785988004637995-12279-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785988004637995-12279-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785988004637995-12279-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785988004637995-12279-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37931
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785988004637995-12279-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45911
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37931
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785988004637995-12279-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785988004637995-12279-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785988004637995-12279-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:51:09.972553 9407 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:51:09.972820 9407 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:51:09.972865 9407 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:51:09.974058 9407 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 03:51:09.974129 9407 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:51:09.974152 9407 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:51:09.974174 9407 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 03:51:09.974205 9407 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 03:51:09.975699 9407 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45911
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785988004637995-12279-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785988004637995-12279-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37931
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785988004637995-12279-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37931
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785988004637995-12279-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9407
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785988004637995-12279-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:51:09.975989 9407 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:51:09.976226 9407 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:51:09.981554 9417 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:51:09.981560 9414 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:51:09.981565 9413 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:51:09.981685 9407 server_base.cc:1034] running on GCE node
I20260806 03:51:09.982039 9407 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:51:09.982395 9407 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:51:09.983569 9407 hybrid_clock.cc:648] HybridClock initialized: now 1785988269983545 us; error 38 us; skew 500 ppm
Aug 06 03:51:09 dist-test-slave-6ntk krb5kdc[9386](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988269, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:51:09.986595 9407 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 03:51:09.987165 9407 webserver.cc:460] Webserver started at http://127.0.0.1:39689/ using document root <none> and password file <none>
I20260806 03:51:09.987478 9407 fs_manager.cc:359] Metadata directory not provided
I20260806 03:51:09.987560 9407 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:51:09.987730 9407 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:51:09.989048 9407 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785988004637995-12279-0/minicluster-data/master-0/data/instance:
uuid: "8dc03f12e270433b82366c0c364994cf"
format_stamp: "Formatted at 2026-08-06 03:51:09 on dist-test-slave-6ntk"
server_key: "8316aa25cd02b09f6416cd79b8d349fb"
server_key_iv: "9b3da9f1c609a0259b0ab55a81feaef4"
server_key_version: "encryptionkey@0"
I20260806 03:51:09.989481 9407 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785988004637995-12279-0/minicluster-data/master-0/wal/instance:
uuid: "8dc03f12e270433b82366c0c364994cf"
format_stamp: "Formatted at 2026-08-06 03:51:09 on dist-test-slave-6ntk"
server_key: "8316aa25cd02b09f6416cd79b8d349fb"
server_key_iv: "9b3da9f1c609a0259b0ab55a81feaef4"
server_key_version: "encryptionkey@0"
I20260806 03:51:09.991112 9407 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.004s
I20260806 03:51:09.992034 9427 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:51:09.992303 9407 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 03:51:09.992416 9407 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785988004637995-12279-0/minicluster-data/master-0/data,/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785988004637995-12279-0/minicluster-data/master-0/wal
uuid: "8dc03f12e270433b82366c0c364994cf"
format_stamp: "Formatted at 2026-08-06 03:51:09 on dist-test-slave-6ntk"
server_key: "8316aa25cd02b09f6416cd79b8d349fb"
server_key_iv: "9b3da9f1c609a0259b0ab55a81feaef4"
server_key_version: "encryptionkey@0"
I20260806 03:51:09.992506 9407 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785988004637995-12279-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785988004637995-12279-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785988004637995-12279-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:51:10.007710 9407 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:51:10.010286 9407 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:51:10.010466 9407 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:51:10.015259 9407 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37931
I20260806 03:51:10.015318 9493 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37931 every 8 connection(s)
I20260806 03:51:10.015826 9407 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785988004637995-12279-0/minicluster-data/master-0/data/info.pb
I20260806 03:51:10.017009 9494 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:51:10.020426 9494 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8dc03f12e270433b82366c0c364994cf: Bootstrap starting.
I20260806 03:51:10.021332 9494 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8dc03f12e270433b82366c0c364994cf: Neither blocks nor log segments found. Creating new log.
I20260806 03:51:10.021397 12279 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskToVWRe/build/release/bin/kudu as pid 9407
I20260806 03:51:10.021528 12279 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785988004637995-12279-0/minicluster-data/master-0/wal/instance
I20260806 03:51:10.021773 12279 external_mini_cluster.cc:1428] Setting key a93c800fe7289ab54e3ce75392f963d1
I20260806 03:51:10.021750 9494 log.cc:824] T 00000000000000000000000000000000 P 8dc03f12e270433b82366c0c364994cf: Log is configured to *not* fsync() on all Append() calls
I20260806 03:51:10.023700 9494 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8dc03f12e270433b82366c0c364994cf: No bootstrap required, opened a new log
I20260806 03:51:10.026153 9494 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8dc03f12e270433b82366c0c364994cf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8dc03f12e270433b82366c0c364994cf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37931 } }
I20260806 03:51:10.026350 9494 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8dc03f12e270433b82366c0c364994cf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:51:10.026384 9494 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8dc03f12e270433b82366c0c364994cf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8dc03f12e270433b82366c0c364994cf, State: Initialized, Role: FOLLOWER
I20260806 03:51:10.026532 9494 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8dc03f12e270433b82366c0c364994cf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8dc03f12e270433b82366c0c364994cf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37931 } }
I20260806 03:51:10.026616 9494 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8dc03f12e270433b82366c0c364994cf [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 03:51:10.026648 9494 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8dc03f12e270433b82366c0c364994cf [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 03:51:10.026695 9494 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8dc03f12e270433b82366c0c364994cf [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:51:10.027375 9494 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8dc03f12e270433b82366c0c364994cf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8dc03f12e270433b82366c0c364994cf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37931 } }
I20260806 03:51:10.027515 9494 leader_election.cc:302] T 00000000000000000000000000000000 P 8dc03f12e270433b82366c0c364994cf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8dc03f12e270433b82366c0c364994cf; no voters:
I20260806 03:51:10.027732 9494 leader_election.cc:288] T 00000000000000000000000000000000 P 8dc03f12e270433b82366c0c364994cf [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 03:51:10.027807 9499 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8dc03f12e270433b82366c0c364994cf [term 1 FOLLOWER]: Leader election won for term 1
I20260806 03:51:10.027976 9499 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8dc03f12e270433b82366c0c364994cf [term 1 LEADER]: Becoming Leader. State: Replica: 8dc03f12e270433b82366c0c364994cf, State: Running, Role: LEADER
I20260806 03:51:10.028039 9494 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8dc03f12e270433b82366c0c364994cf [sys.catalog]: configured and running, proceeding with master startup.
I20260806 03:51:10.028111 9499 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8dc03f12e270433b82366c0c364994cf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8dc03f12e270433b82366c0c364994cf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37931 } }
I20260806 03:51:10.028594 9500 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8dc03f12e270433b82366c0c364994cf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8dc03f12e270433b82366c0c364994cf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8dc03f12e270433b82366c0c364994cf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37931 } } }
I20260806 03:51:10.028609 9501 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8dc03f12e270433b82366c0c364994cf [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8dc03f12e270433b82366c0c364994cf. Latest consensus state: current_term: 1 leader_uuid: "8dc03f12e270433b82366c0c364994cf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8dc03f12e270433b82366c0c364994cf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37931 } } }
I20260806 03:51:10.028779 9501 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8dc03f12e270433b82366c0c364994cf [sys.catalog]: This master's current role is: LEADER
I20260806 03:51:10.028764 9500 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8dc03f12e270433b82366c0c364994cf [sys.catalog]: This master's current role is: LEADER
I20260806 03:51:10.029099 9498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:10.022436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.193:59415 (local address 127.0.0.1:37931)
0806 03:51:10.022632 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0806 03:51:10.022644 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 03:51:10.022682 (+ 38us) server_negotiation.cc:407] Connection header received
0806 03:51:10.024011 (+ 1329us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:51:10.024021 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:51:10.024092 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:51:10.024257 (+ 165us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:51:10.024660 (+ 403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:10.025597 (+ 937us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:10.026761 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:10.027222 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:10.027311 (+ 89us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:10.028148 (+ 837us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:51:10.028152 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:51:10.028152 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 03:51:10.028373 (+ 221us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:51:10.028414 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 03:51:10.028430 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:51:10.028528 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0806 03:51:10.028827 (+ 299us) server_negotiation.cc:299] Negotiation successful
0806 03:51:10.028867 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":67,"threads_started":1}
I20260806 03:51:10.029112 9514 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 03:51:10.030587 12279 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 03:51:10.030658 9514 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 03:51:10.032987 9514 catalog_manager.cc:1347] Generated new cluster ID: 532530d5e4444dfd92b6c8a5d654c82d
I20260806 03:51:10.033041 9514 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 03:51:10.046133 9498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:10.032170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.193:43455 (local address 127.0.0.1:37931)
0806 03:51:10.032205 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0806 03:51:10.032207 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:51:10.042121 (+ 9914us) server_negotiation.cc:407] Connection header received
0806 03:51:10.042217 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:51:10.042218 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:51:10.042231 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:51:10.042291 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:51:10.042501 (+ 210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:10.043307 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:10.044492 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:10.044838 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:10.044916 (+ 78us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:10.045805 (+ 889us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:51:10.045807 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:51:10.045807 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 03:51:10.045814 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:51:10.045832 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 03:51:10.045836 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:51:10.045949 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0806 03:51:10.046019 (+ 70us) server_negotiation.cc:299] Negotiation successful
0806 03:51:10.046032 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260806 03:51:10.051950 9514 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 03:51:10.052764 9514 catalog_manager.cc:1504] Loading token signing keys...
I20260806 03:51:10.062062 9514 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8dc03f12e270433b82366c0c364994cf: Generated new TSK 0
I20260806 03:51:10.062323 9514 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 03:51:10.073378 12279 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskToVWRe/build/release/bin/kudu with pid 9407
W20260806 03:51:10.085255 9383 connection.cc:535] client connection to 127.0.0.1:37931 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:37931): Connection reset by peer (error 104)
2026-08-06T03:51:10Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1537 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785988004637995-12279-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 03:51:10 dist-test-slave-6ntk krb5kdc[9539](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 03:51:10 dist-test-slave-6ntk krb5kdc[9539](info): set up 2 sockets
Aug 06 03:51:10 dist-test-slave-6ntk krb5kdc[9539](info): commencing operation
krb5kdc: starting...
W20260806 03:51:11.290233 12279 mini_kdc.cc:121] Time spent starting KDC: real 1.191s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785988004637995-12279-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785988004637995-12279-0/krb5kdc/test-user.keytab.
Aug 06 03:51:11 dist-test-slave-6ntk krb5kdc[9539](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988271, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T03:51:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T03:51:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785988004637995-12279-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785988004637995-12279-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 03:51:11.426077 12279 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskToVWRe/build/release/bin/kudu
/tmp/dist-test-taskToVWRe/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785988004637995-12279-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785988004637995-12279-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785988004637995-12279-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785988004637995-12279-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46671
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785988004637995-12279-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40735
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46671
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785988004637995-12279-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785988004637995-12279-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785988004637995-12279-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:51:11.510730 9555 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:51:11.511030 9555 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:51:11.511075 9555 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:51:11.512261 9555 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 03:51:11.512341 9555 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:51:11.512365 9555 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:51:11.512387 9555 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 03:51:11.512418 9555 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 03:51:11.513994 9555 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40735
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785988004637995-12279-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785988004637995-12279-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46671
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785988004637995-12279-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46671
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785988004637995-12279-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9555
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785988004637995-12279-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:51:11.514297 9555 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:51:11.514689 9555 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:51:11.520079 9561 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:51:11.520079 9565 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:51:11.520114 9563 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:51:11.520648 9555 server_base.cc:1034] running on GCE node
I20260806 03:51:11.520915 9555 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:51:11.521279 9555 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:51:11.522467 9555 hybrid_clock.cc:648] HybridClock initialized: now 1785988271522445 us; error 44 us; skew 500 ppm
Aug 06 03:51:11 dist-test-slave-6ntk krb5kdc[9539](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988271, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:51:11.528944 9555 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 03:51:11.529451 9555 webserver.cc:460] Webserver started at http://127.0.0.1:37791/ using document root <none> and password file <none>
I20260806 03:51:11.529713 9555 fs_manager.cc:359] Metadata directory not provided
I20260806 03:51:11.529794 9555 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:51:11.529964 9555 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:51:11.531067 9555 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785988004637995-12279-0/minicluster-data/master-0/data/instance:
uuid: "200f115949914c358e4eae90fe0ff49b"
format_stamp: "Formatted at 2026-08-06 03:51:11 on dist-test-slave-6ntk"
server_key: "52705fffee38ef2519e363a8c6c26e15"
server_key_iv: "b34de43497076d1fb6382ca4d8b202cd"
server_key_version: "encryptionkey@0"
I20260806 03:51:11.531503 9555 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785988004637995-12279-0/minicluster-data/master-0/wal/instance:
uuid: "200f115949914c358e4eae90fe0ff49b"
format_stamp: "Formatted at 2026-08-06 03:51:11 on dist-test-slave-6ntk"
server_key: "52705fffee38ef2519e363a8c6c26e15"
server_key_iv: "b34de43497076d1fb6382ca4d8b202cd"
server_key_version: "encryptionkey@0"
I20260806 03:51:11.532984 9555 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 03:51:11.533818 9575 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:51:11.534051 9555 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 03:51:11.534159 9555 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785988004637995-12279-0/minicluster-data/master-0/data,/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785988004637995-12279-0/minicluster-data/master-0/wal
uuid: "200f115949914c358e4eae90fe0ff49b"
format_stamp: "Formatted at 2026-08-06 03:51:11 on dist-test-slave-6ntk"
server_key: "52705fffee38ef2519e363a8c6c26e15"
server_key_iv: "b34de43497076d1fb6382ca4d8b202cd"
server_key_version: "encryptionkey@0"
I20260806 03:51:11.534245 9555 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785988004637995-12279-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785988004637995-12279-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785988004637995-12279-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:51:11.537616 9555 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:51:11.540035 9555 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:51:11.540205 9555 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:51:11.544581 9555 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46671
I20260806 03:51:11.544621 9642 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46671 every 8 connection(s)
I20260806 03:51:11.544981 9555 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785988004637995-12279-0/minicluster-data/master-0/data/info.pb
I20260806 03:51:11.545990 9643 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:51:11.548406 9643 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 200f115949914c358e4eae90fe0ff49b: Bootstrap starting.
I20260806 03:51:11.549016 9643 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 200f115949914c358e4eae90fe0ff49b: Neither blocks nor log segments found. Creating new log.
I20260806 03:51:11.549345 9643 log.cc:824] T 00000000000000000000000000000000 P 200f115949914c358e4eae90fe0ff49b: Log is configured to *not* fsync() on all Append() calls
I20260806 03:51:11.550356 9643 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 200f115949914c358e4eae90fe0ff49b: No bootstrap required, opened a new log
I20260806 03:51:11.551843 12279 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskToVWRe/build/release/bin/kudu as pid 9555
I20260806 03:51:11.552052 12279 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785988004637995-12279-0/minicluster-data/master-0/wal/instance
I20260806 03:51:11.552214 9643 raft_consensus.cc:348] T 00000000000000000000000000000000 P 200f115949914c358e4eae90fe0ff49b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "200f115949914c358e4eae90fe0ff49b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46671 } }
I20260806 03:51:11.552341 12279 external_mini_cluster.cc:1428] Setting key 785a75d5c412c50f33c94982ece8443f
I20260806 03:51:11.552376 9643 raft_consensus.cc:374] T 00000000000000000000000000000000 P 200f115949914c358e4eae90fe0ff49b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:51:11.552402 9643 raft_consensus.cc:729] T 00000000000000000000000000000000 P 200f115949914c358e4eae90fe0ff49b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 200f115949914c358e4eae90fe0ff49b, State: Initialized, Role: FOLLOWER
I20260806 03:51:11.552589 9643 consensus_queue.cc:260] T 00000000000000000000000000000000 P 200f115949914c358e4eae90fe0ff49b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "200f115949914c358e4eae90fe0ff49b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46671 } }
I20260806 03:51:11.552699 9643 raft_consensus.cc:388] T 00000000000000000000000000000000 P 200f115949914c358e4eae90fe0ff49b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 03:51:11.552744 9643 raft_consensus.cc:482] T 00000000000000000000000000000000 P 200f115949914c358e4eae90fe0ff49b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 03:51:11.552858 9643 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 200f115949914c358e4eae90fe0ff49b [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:51:11.553597 9643 raft_consensus.cc:504] T 00000000000000000000000000000000 P 200f115949914c358e4eae90fe0ff49b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "200f115949914c358e4eae90fe0ff49b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46671 } }
I20260806 03:51:11.553727 9643 leader_election.cc:302] T 00000000000000000000000000000000 P 200f115949914c358e4eae90fe0ff49b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 200f115949914c358e4eae90fe0ff49b; no voters:
I20260806 03:51:11.554005 9643 leader_election.cc:288] T 00000000000000000000000000000000 P 200f115949914c358e4eae90fe0ff49b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 03:51:11.554026 9649 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 200f115949914c358e4eae90fe0ff49b [term 1 FOLLOWER]: Leader election won for term 1
I20260806 03:51:11.554368 9643 sys_catalog.cc:564] T 00000000000000000000000000000000 P 200f115949914c358e4eae90fe0ff49b [sys.catalog]: configured and running, proceeding with master startup.
I20260806 03:51:11.554616 9649 raft_consensus.cc:686] T 00000000000000000000000000000000 P 200f115949914c358e4eae90fe0ff49b [term 1 LEADER]: Becoming Leader. State: Replica: 200f115949914c358e4eae90fe0ff49b, State: Running, Role: LEADER
I20260806 03:51:11.554714 9649 consensus_queue.cc:237] T 00000000000000000000000000000000 P 200f115949914c358e4eae90fe0ff49b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "200f115949914c358e4eae90fe0ff49b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46671 } }
I20260806 03:51:11.555220 9648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:11.552960 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.193:57929 (local address 127.0.0.1:46671)
0806 03:51:11.553182 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0806 03:51:11.553191 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 03:51:11.553223 (+ 32us) server_negotiation.cc:407] Connection header received
0806 03:51:11.553589 (+ 366us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:51:11.553595 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:51:11.553622 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:51:11.553809 (+ 187us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:51:11.554625 (+ 816us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:51:11.554630 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:51:11.554631 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 03:51:11.554636 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:51:11.554689 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 03:51:11.554704 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:51:11.554780 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0806 03:51:11.554921 (+ 141us) server_negotiation.cc:299] Negotiation successful
0806 03:51:11.554980 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":108,"threads_started":1}
I20260806 03:51:11.555382 9650 sys_catalog.cc:455] T 00000000000000000000000000000000 P 200f115949914c358e4eae90fe0ff49b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "200f115949914c358e4eae90fe0ff49b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "200f115949914c358e4eae90fe0ff49b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46671 } } }
I20260806 03:51:11.555495 9650 sys_catalog.cc:458] T 00000000000000000000000000000000 P 200f115949914c358e4eae90fe0ff49b [sys.catalog]: This master's current role is: LEADER
I20260806 03:51:11.555652 9652 sys_catalog.cc:455] T 00000000000000000000000000000000 P 200f115949914c358e4eae90fe0ff49b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 200f115949914c358e4eae90fe0ff49b. Latest consensus state: current_term: 1 leader_uuid: "200f115949914c358e4eae90fe0ff49b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "200f115949914c358e4eae90fe0ff49b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46671 } } }
I20260806 03:51:11.555965 9665 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 03:51:11.556010 9652 sys_catalog.cc:458] T 00000000000000000000000000000000 P 200f115949914c358e4eae90fe0ff49b [sys.catalog]: This master's current role is: LEADER
I20260806 03:51:11.556424 12279 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 03:51:11.558174 9665 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 03:51:11.559885 9665 catalog_manager.cc:1347] Generated new cluster ID: 6ac3b60c4656409792e92dbc19b9e54b
I20260806 03:51:11.559947 9665 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 03:51:11.563393 9648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:11.557958 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.193:51709 (local address 127.0.0.1:46671)
0806 03:51:11.558018 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0806 03:51:11.558020 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:51:11.562145 (+ 4125us) server_negotiation.cc:407] Connection header received
0806 03:51:11.562236 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:51:11.562237 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:51:11.562246 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:51:11.562370 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:51:11.563096 (+ 726us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:51:11.563098 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:51:11.563099 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 03:51:11.563104 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:51:11.563121 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 03:51:11.563126 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:51:11.563189 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0806 03:51:11.563264 (+ 75us) server_negotiation.cc:299] Negotiation successful
0806 03:51:11.563279 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260806 03:51:11.583380 9665 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 03:51:11.583611 9665 catalog_manager.cc:1504] Loading token signing keys...
I20260806 03:51:11.588280 9665 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 200f115949914c358e4eae90fe0ff49b: Generated new TSK 0
I20260806 03:51:11.588528 9665 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 03:51:11.590565 12279 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskToVWRe/build/release/bin/kudu with pid 9555
2026-08-06T03:51:11Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1518 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785988004637995-12279-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 03:51:11 dist-test-slave-6ntk krb5kdc[9686](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 03:51:11 dist-test-slave-6ntk krb5kdc[9686](info): set up 2 sockets
Aug 06 03:51:11 dist-test-slave-6ntk krb5kdc[9686](info): commencing operation
krb5kdc: starting...
W20260806 03:51:12.832294 12279 mini_kdc.cc:121] Time spent starting KDC: real 1.214s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785988004637995-12279-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785988004637995-12279-0/krb5kdc/test-user.keytab.
Aug 06 03:51:12 dist-test-slave-6ntk krb5kdc[9686](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988272, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T03:51:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T03:51:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785988004637995-12279-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785988004637995-12279-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 03:51:12.962162 12279 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskToVWRe/build/release/bin/kudu
/tmp/dist-test-taskToVWRe/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785988004637995-12279-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785988004637995-12279-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785988004637995-12279-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785988004637995-12279-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35181
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785988004637995-12279-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46103
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35181
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785988004637995-12279-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785988004637995-12279-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785988004637995-12279-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:51:13.048818 9701 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:51:13.049029 9701 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:51:13.049053 9701 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:51:13.050206 9701 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 03:51:13.050247 9701 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:51:13.050261 9701 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:51:13.050275 9701 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 03:51:13.050287 9701 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 03:51:13.051685 9701 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46103
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785988004637995-12279-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785988004637995-12279-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35181
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785988004637995-12279-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35181
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785988004637995-12279-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9701
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785988004637995-12279-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:51:13.051883 9701 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:51:13.052062 9701 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:51:13.057271 9709 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:51:13.057283 9710 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:51:13.057582 9701 server_base.cc:1034] running on GCE node
W20260806 03:51:13.057349 9712 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:51:13.057941 9701 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:51:13.058261 9701 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:51:13.059435 9701 hybrid_clock.cc:648] HybridClock initialized: now 1785988273059424 us; error 37 us; skew 500 ppm
Aug 06 03:51:13 dist-test-slave-6ntk krb5kdc[9686](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988273, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:51:13.062076 9701 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 03:51:13.062485 9701 webserver.cc:460] Webserver started at http://127.0.0.1:44695/ using document root <none> and password file <none>
I20260806 03:51:13.062670 9701 fs_manager.cc:359] Metadata directory not provided
I20260806 03:51:13.062711 9701 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:51:13.062808 9701 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:51:13.063834 9701 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785988004637995-12279-0/minicluster-data/master-0/data/instance:
uuid: "5106735c44b146c0a5dfeff5f9b829d8"
format_stamp: "Formatted at 2026-08-06 03:51:13 on dist-test-slave-6ntk"
server_key: "04bca1508ca3116f672cb01c0b63ab3b"
server_key_iv: "7f606c8632435013d64e2945a1fbd251"
server_key_version: "encryptionkey@0"
I20260806 03:51:13.064142 9701 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785988004637995-12279-0/minicluster-data/master-0/wal/instance:
uuid: "5106735c44b146c0a5dfeff5f9b829d8"
format_stamp: "Formatted at 2026-08-06 03:51:13 on dist-test-slave-6ntk"
server_key: "04bca1508ca3116f672cb01c0b63ab3b"
server_key_iv: "7f606c8632435013d64e2945a1fbd251"
server_key_version: "encryptionkey@0"
I20260806 03:51:13.065543 9701 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 03:51:13.066395 9719 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:51:13.066576 9701 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260806 03:51:13.066674 9701 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785988004637995-12279-0/minicluster-data/master-0/data,/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785988004637995-12279-0/minicluster-data/master-0/wal
uuid: "5106735c44b146c0a5dfeff5f9b829d8"
format_stamp: "Formatted at 2026-08-06 03:51:13 on dist-test-slave-6ntk"
server_key: "04bca1508ca3116f672cb01c0b63ab3b"
server_key_iv: "7f606c8632435013d64e2945a1fbd251"
server_key_version: "encryptionkey@0"
I20260806 03:51:13.066736 9701 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785988004637995-12279-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785988004637995-12279-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785988004637995-12279-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:51:13.086845 9701 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:51:13.089099 9701 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:51:13.089221 9701 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:51:13.093254 9701 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35181
I20260806 03:51:13.093339 9785 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35181 every 8 connection(s)
I20260806 03:51:13.093713 9701 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785988004637995-12279-0/minicluster-data/master-0/data/info.pb
I20260806 03:51:13.094720 9786 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:51:13.097075 9786 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5106735c44b146c0a5dfeff5f9b829d8: Bootstrap starting.
I20260806 03:51:13.097666 9786 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5106735c44b146c0a5dfeff5f9b829d8: Neither blocks nor log segments found. Creating new log.
I20260806 03:51:13.098006 9786 log.cc:824] T 00000000000000000000000000000000 P 5106735c44b146c0a5dfeff5f9b829d8: Log is configured to *not* fsync() on all Append() calls
I20260806 03:51:13.098198 12279 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskToVWRe/build/release/bin/kudu as pid 9701
I20260806 03:51:13.098366 12279 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785988004637995-12279-0/minicluster-data/master-0/wal/instance
I20260806 03:51:13.098582 12279 external_mini_cluster.cc:1428] Setting key 2e968b7aa6893b454d069a3621498111
I20260806 03:51:13.098840 9786 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5106735c44b146c0a5dfeff5f9b829d8: No bootstrap required, opened a new log
I20260806 03:51:13.100597 9786 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5106735c44b146c0a5dfeff5f9b829d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5106735c44b146c0a5dfeff5f9b829d8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35181 } }
I20260806 03:51:13.100761 9786 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5106735c44b146c0a5dfeff5f9b829d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:51:13.100837 9786 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5106735c44b146c0a5dfeff5f9b829d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5106735c44b146c0a5dfeff5f9b829d8, State: Initialized, Role: FOLLOWER
I20260806 03:51:13.101013 9786 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5106735c44b146c0a5dfeff5f9b829d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5106735c44b146c0a5dfeff5f9b829d8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35181 } }
I20260806 03:51:13.101111 9786 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5106735c44b146c0a5dfeff5f9b829d8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 03:51:13.101157 9786 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5106735c44b146c0a5dfeff5f9b829d8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 03:51:13.101205 9786 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5106735c44b146c0a5dfeff5f9b829d8 [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:51:13.102176 9786 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5106735c44b146c0a5dfeff5f9b829d8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5106735c44b146c0a5dfeff5f9b829d8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35181 } }
I20260806 03:51:13.102300 9786 leader_election.cc:302] T 00000000000000000000000000000000 P 5106735c44b146c0a5dfeff5f9b829d8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5106735c44b146c0a5dfeff5f9b829d8; no voters:
I20260806 03:51:13.102504 9786 leader_election.cc:288] T 00000000000000000000000000000000 P 5106735c44b146c0a5dfeff5f9b829d8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 03:51:13.102525 9794 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5106735c44b146c0a5dfeff5f9b829d8 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 03:51:13.102643 9794 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5106735c44b146c0a5dfeff5f9b829d8 [term 1 LEADER]: Becoming Leader. State: Replica: 5106735c44b146c0a5dfeff5f9b829d8, State: Running, Role: LEADER
I20260806 03:51:13.102773 9786 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5106735c44b146c0a5dfeff5f9b829d8 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 03:51:13.102825 9794 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5106735c44b146c0a5dfeff5f9b829d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5106735c44b146c0a5dfeff5f9b829d8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35181 } }
I20260806 03:51:13.103262 9796 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5106735c44b146c0a5dfeff5f9b829d8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5106735c44b146c0a5dfeff5f9b829d8. Latest consensus state: current_term: 1 leader_uuid: "5106735c44b146c0a5dfeff5f9b829d8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5106735c44b146c0a5dfeff5f9b829d8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35181 } } }
I20260806 03:51:13.103367 9796 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5106735c44b146c0a5dfeff5f9b829d8 [sys.catalog]: This master's current role is: LEADER
I20260806 03:51:13.103579 9795 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5106735c44b146c0a5dfeff5f9b829d8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5106735c44b146c0a5dfeff5f9b829d8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5106735c44b146c0a5dfeff5f9b829d8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35181 } } }
I20260806 03:51:13.103869 9795 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5106735c44b146c0a5dfeff5f9b829d8 [sys.catalog]: This master's current role is: LEADER
I20260806 03:51:13.104100 9803 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 03:51:13.104804 9803 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 03:51:13.106699 9803 catalog_manager.cc:1347] Generated new cluster ID: 9fc0bdc02501498fab45b1247e31d4b7
I20260806 03:51:13.106750 9803 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 03:51:13 dist-test-slave-6ntk krb5kdc[9686](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988272, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260806 03:51:13.111650 9793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:13.099160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.193:49409 (local address 127.0.0.1:35181)
0806 03:51:13.099350 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0806 03:51:13.099359 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 03:51:13.099399 (+ 40us) server_negotiation.cc:407] Connection header received
0806 03:51:13.100581 (+ 1182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:51:13.100588 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:51:13.100641 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:51:13.100790 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:51:13.101563 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:13.102764 (+ 1201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:13.103848 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:13.104470 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:13.104577 (+ 107us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:13.107588 (+ 3011us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:51:13.107592 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:51:13.107592 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:51:13.107604 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:51:13.109577 (+ 1973us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:51:13.110067 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:51:13.110069 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:51:13.110070 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:51:13.110194 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:51:13.110414 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:51:13.110415 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:51:13.110415 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:51:13.110705 (+ 290us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:51:13.111254 (+ 549us) server_negotiation.cc:1035] Waiting for connection context
0806 03:51:13.111336 (+ 82us) server_negotiation.cc:299] Negotiation successful
0806 03:51:13.111436 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":83,"threads_started":1}
I20260806 03:51:13.112756 12279 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 03:51:13.126093 9793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:13.114295 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.193:37535 (local address 127.0.0.1:35181)
0806 03:51:13.114330 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0806 03:51:13.114332 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:51:13.118063 (+ 3731us) server_negotiation.cc:407] Connection header received
0806 03:51:13.118129 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:51:13.118130 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:51:13.118141 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:51:13.118200 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:51:13.118851 (+ 651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:13.119607 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:13.120687 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:13.121011 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:13.121072 (+ 61us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:13.122553 (+ 1481us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:51:13.122555 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:51:13.122555 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:51:13.122562 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:51:13.124800 (+ 2238us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:51:13.125227 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:51:13.125227 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:51:13.125228 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:51:13.125284 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:51:13.125521 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:51:13.125522 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:51:13.125522 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:51:13.125643 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:51:13.125748 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0806 03:51:13.125938 (+ 190us) server_negotiation.cc:299] Negotiation successful
0806 03:51:13.126001 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260806 03:51:13.129595 9803 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 03:51:13.130172 9803 catalog_manager.cc:1504] Loading token signing keys...
I20260806 03:51:13.136693 9803 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5106735c44b146c0a5dfeff5f9b829d8: Generated new TSK 0
I20260806 03:51:13.136884 9803 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 03:51:13.138887 12279 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskToVWRe/build/release/bin/kudu with pid 9701
W20260806 03:51:13.152132 9682 connection.cc:535] client connection to 127.0.0.1:35181 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:35181): Connection reset by peer (error 104)
2026-08-06T03:51:13Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1548 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785988004637995-12279-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 03:51:13 dist-test-slave-6ntk krb5kdc[9830](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 03:51:13 dist-test-slave-6ntk krb5kdc[9830](info): set up 2 sockets
Aug 06 03:51:13 dist-test-slave-6ntk krb5kdc[9830](info): commencing operation
krb5kdc: starting...
W20260806 03:51:14.383426 12279 mini_kdc.cc:121] Time spent starting KDC: real 1.217s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785988004637995-12279-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785988004637995-12279-0/krb5kdc/test-user.keytab.
Aug 06 03:51:14 dist-test-slave-6ntk krb5kdc[9830](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988274, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T03:51:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T03:51:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785988004637995-12279-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785988004637995-12279-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 03:51:14.518080 12279 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskToVWRe/build/release/bin/kudu
/tmp/dist-test-taskToVWRe/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785988004637995-12279-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785988004637995-12279-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785988004637995-12279-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785988004637995-12279-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45945
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785988004637995-12279-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35023
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45945
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785988004637995-12279-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785988004637995-12279-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785988004637995-12279-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:51:14.605151 9849 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:51:14.605422 9849 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:51:14.605466 9849 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:51:14.606739 9849 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 03:51:14.606819 9849 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:51:14.606844 9849 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:51:14.606866 9849 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 03:51:14.606899 9849 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 03:51:14.608441 9849 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35023
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785988004637995-12279-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785988004637995-12279-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45945
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785988004637995-12279-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45945
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785988004637995-12279-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9849
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785988004637995-12279-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:51:14.608759 9849 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:51:14.609014 9849 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:51:14.614295 9858 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:51:14.614442 9855 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:51:14.614436 9856 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:51:14.614622 9849 server_base.cc:1034] running on GCE node
I20260806 03:51:14.614905 9849 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:51:14.615250 9849 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:51:14.616454 9849 hybrid_clock.cc:648] HybridClock initialized: now 1785988274616429 us; error 39 us; skew 500 ppm
Aug 06 03:51:14 dist-test-slave-6ntk krb5kdc[9830](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988274, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:51:14.619390 9849 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 03:51:14.619936 9849 webserver.cc:460] Webserver started at http://127.0.0.1:38037/ using document root <none> and password file <none>
I20260806 03:51:14.620201 9849 fs_manager.cc:359] Metadata directory not provided
I20260806 03:51:14.620253 9849 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:51:14.620427 9849 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:51:14.621608 9849 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785988004637995-12279-0/minicluster-data/master-0/data/instance:
uuid: "cac94acb529946ef93c2492a4ba8db12"
format_stamp: "Formatted at 2026-08-06 03:51:14 on dist-test-slave-6ntk"
server_key: "42ffd2500e4c8b99453e20414a54050e"
server_key_iv: "fddc652469919fff54ff574347d1f991"
server_key_version: "encryptionkey@0"
I20260806 03:51:14.622018 9849 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785988004637995-12279-0/minicluster-data/master-0/wal/instance:
uuid: "cac94acb529946ef93c2492a4ba8db12"
format_stamp: "Formatted at 2026-08-06 03:51:14 on dist-test-slave-6ntk"
server_key: "42ffd2500e4c8b99453e20414a54050e"
server_key_iv: "fddc652469919fff54ff574347d1f991"
server_key_version: "encryptionkey@0"
I20260806 03:51:14.623497 9849 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 03:51:14.624330 9867 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:51:14.624548 9849 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 03:51:14.624655 9849 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785988004637995-12279-0/minicluster-data/master-0/data,/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785988004637995-12279-0/minicluster-data/master-0/wal
uuid: "cac94acb529946ef93c2492a4ba8db12"
format_stamp: "Formatted at 2026-08-06 03:51:14 on dist-test-slave-6ntk"
server_key: "42ffd2500e4c8b99453e20414a54050e"
server_key_iv: "fddc652469919fff54ff574347d1f991"
server_key_version: "encryptionkey@0"
I20260806 03:51:14.624734 9849 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785988004637995-12279-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785988004637995-12279-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785988004637995-12279-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:51:14.641024 9849 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:51:14.643792 9849 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:51:14.643925 9849 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:51:14.648763 9849 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45945
I20260806 03:51:14.648857 9932 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45945 every 8 connection(s)
I20260806 03:51:14.649267 9849 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785988004637995-12279-0/minicluster-data/master-0/data/info.pb
I20260806 03:51:14.650331 9934 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:51:14.652825 9934 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cac94acb529946ef93c2492a4ba8db12: Bootstrap starting.
I20260806 03:51:14.653447 9934 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cac94acb529946ef93c2492a4ba8db12: Neither blocks nor log segments found. Creating new log.
I20260806 03:51:14.653764 9934 log.cc:824] T 00000000000000000000000000000000 P cac94acb529946ef93c2492a4ba8db12: Log is configured to *not* fsync() on all Append() calls
I20260806 03:51:14.654448 12279 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskToVWRe/build/release/bin/kudu as pid 9849
I20260806 03:51:14.654593 12279 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785988004637995-12279-0/minicluster-data/master-0/wal/instance
I20260806 03:51:14.654654 9934 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cac94acb529946ef93c2492a4ba8db12: No bootstrap required, opened a new log
I20260806 03:51:14.654831 12279 external_mini_cluster.cc:1428] Setting key 68d5f87a2466a1b36f140a6b607e2f24
I20260806 03:51:14.656888 9934 raft_consensus.cc:348] T 00000000000000000000000000000000 P cac94acb529946ef93c2492a4ba8db12 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cac94acb529946ef93c2492a4ba8db12" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45945 } }
I20260806 03:51:14.657120 9934 raft_consensus.cc:374] T 00000000000000000000000000000000 P cac94acb529946ef93c2492a4ba8db12 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:51:14.657158 9934 raft_consensus.cc:729] T 00000000000000000000000000000000 P cac94acb529946ef93c2492a4ba8db12 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cac94acb529946ef93c2492a4ba8db12, State: Initialized, Role: FOLLOWER
I20260806 03:51:14.657291 9934 consensus_queue.cc:260] T 00000000000000000000000000000000 P cac94acb529946ef93c2492a4ba8db12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cac94acb529946ef93c2492a4ba8db12" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45945 } }
I20260806 03:51:14.657389 9934 raft_consensus.cc:388] T 00000000000000000000000000000000 P cac94acb529946ef93c2492a4ba8db12 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 03:51:14.657428 9934 raft_consensus.cc:482] T 00000000000000000000000000000000 P cac94acb529946ef93c2492a4ba8db12 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 03:51:14.657542 9934 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cac94acb529946ef93c2492a4ba8db12 [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:51:14.658289 9934 raft_consensus.cc:504] T 00000000000000000000000000000000 P cac94acb529946ef93c2492a4ba8db12 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cac94acb529946ef93c2492a4ba8db12" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45945 } }
I20260806 03:51:14.658433 9934 leader_election.cc:302] T 00000000000000000000000000000000 P cac94acb529946ef93c2492a4ba8db12 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cac94acb529946ef93c2492a4ba8db12; no voters:
I20260806 03:51:14.658680 9934 leader_election.cc:288] T 00000000000000000000000000000000 P cac94acb529946ef93c2492a4ba8db12 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 03:51:14.658712 9939 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cac94acb529946ef93c2492a4ba8db12 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 03:51:14.658835 9939 raft_consensus.cc:686] T 00000000000000000000000000000000 P cac94acb529946ef93c2492a4ba8db12 [term 1 LEADER]: Becoming Leader. State: Replica: cac94acb529946ef93c2492a4ba8db12, State: Running, Role: LEADER
I20260806 03:51:14.658985 9939 consensus_queue.cc:237] T 00000000000000000000000000000000 P cac94acb529946ef93c2492a4ba8db12 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cac94acb529946ef93c2492a4ba8db12" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45945 } }
I20260806 03:51:14.658993 9934 sys_catalog.cc:564] T 00000000000000000000000000000000 P cac94acb529946ef93c2492a4ba8db12 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 03:51:14.660346 9941 sys_catalog.cc:455] T 00000000000000000000000000000000 P cac94acb529946ef93c2492a4ba8db12 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cac94acb529946ef93c2492a4ba8db12" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cac94acb529946ef93c2492a4ba8db12" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45945 } } }
I20260806 03:51:14.660447 9941 sys_catalog.cc:458] T 00000000000000000000000000000000 P cac94acb529946ef93c2492a4ba8db12 [sys.catalog]: This master's current role is: LEADER
I20260806 03:51:14.660784 9943 sys_catalog.cc:455] T 00000000000000000000000000000000 P cac94acb529946ef93c2492a4ba8db12 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cac94acb529946ef93c2492a4ba8db12. Latest consensus state: current_term: 1 leader_uuid: "cac94acb529946ef93c2492a4ba8db12" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cac94acb529946ef93c2492a4ba8db12" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45945 } } }
I20260806 03:51:14.660876 9943 sys_catalog.cc:458] T 00000000000000000000000000000000 P cac94acb529946ef93c2492a4ba8db12 [sys.catalog]: This master's current role is: LEADER
W20260806 03:51:14.661027 9959 catalog_manager.cc:1558] T 00000000000000000000000000000000 P cac94acb529946ef93c2492a4ba8db12: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 03:51:14.661118 9959 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 03:51:14.661191 9960 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 03:51:14.661438 9960 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 03:51:14.662483 9938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:14.655434 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.193:42105 (local address 127.0.0.1:45945)
0806 03:51:14.655638 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0806 03:51:14.655647 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 03:51:14.655682 (+ 35us) server_negotiation.cc:407] Connection header received
0806 03:51:14.657110 (+ 1428us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:51:14.657121 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:51:14.657195 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:51:14.657380 (+ 185us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:51:14.657668 (+ 288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:14.658867 (+ 1199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:14.659977 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:14.660639 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:14.661111 (+ 472us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:14.661609 (+ 498us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:51:14.661613 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:51:14.661613 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 03:51:14.661828 (+ 215us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:51:14.661905 (+ 77us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 03:51:14.661922 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:51:14.662252 (+ 330us) server_negotiation.cc:1035] Waiting for connection context
0806 03:51:14.662290 (+ 38us) server_negotiation.cc:299] Negotiation successful
0806 03:51:14.662323 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":86,"threads_started":1}
I20260806 03:51:14.663203 9960 catalog_manager.cc:1347] Generated new cluster ID: 4a7eda121d714e998aa1b68aa8456fbd
I20260806 03:51:14.663260 9960 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 03:51:14.663508 12279 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 03:51:14.680855 9960 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 03:51:14.681505 9960 catalog_manager.cc:1504] Loading token signing keys...
I20260806 03:51:14.682518 9938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:14.664978 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.193:45395 (local address 127.0.0.1:45945)
0806 03:51:14.665012 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0806 03:51:14.665013 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 03:51:14.678121 (+ 13108us) server_negotiation.cc:407] Connection header received
0806 03:51:14.678216 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:51:14.678218 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:51:14.678231 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:51:14.678285 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:51:14.678540 (+ 255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:14.679797 (+ 1257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:14.680923 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:14.681190 (+ 267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:14.681256 (+ 66us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:14.682241 (+ 985us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:51:14.682243 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:51:14.682244 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 03:51:14.682249 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:51:14.682266 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 03:51:14.682270 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:51:14.682337 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0806 03:51:14.682425 (+ 88us) server_negotiation.cc:299] Negotiation successful
0806 03:51:14.682438 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":46,"server-negotiator.queue_time_us":15}
I20260806 03:51:14.692019 9960 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cac94acb529946ef93c2492a4ba8db12: Generated new TSK 0
I20260806 03:51:14.692265 9960 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 03:51:14.701144 12279 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskToVWRe/build/release/bin/kudu with pid 9849
W20260806 03:51:14.723572 9827 connection.cc:535] client connection to 127.0.0.1:45945 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:45945): Connection reset by peer (error 104)
2026-08-06T03:51:14Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1571 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785988004637995-12279-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 03:51:14 dist-test-slave-6ntk krb5kdc[9974](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 03:51:14 dist-test-slave-6ntk krb5kdc[9974](info): set up 2 sockets
Aug 06 03:51:14 dist-test-slave-6ntk krb5kdc[9974](info): commencing operation
krb5kdc: starting...
W20260806 03:51:15.955430 12279 mini_kdc.cc:121] Time spent starting KDC: real 1.217s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785988004637995-12279-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785988004637995-12279-0/krb5kdc/test-user.keytab.
Aug 06 03:51:16 dist-test-slave-6ntk krb5kdc[9974](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988276, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T03:51:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T03:51:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785988004637995-12279-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785988004637995-12279-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 03:51:16.089468 12279 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskToVWRe/build/release/bin/kudu
/tmp/dist-test-taskToVWRe/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785988004637995-12279-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785988004637995-12279-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785988004637995-12279-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785988004637995-12279-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39641
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785988004637995-12279-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38969
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39641
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785988004637995-12279-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785988004637995-12279-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785988004637995-12279-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:51:16.173624 9993 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:51:16.173874 9993 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:51:16.173952 9993 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:51:16.175123 9993 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 03:51:16.175197 9993 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:51:16.175222 9993 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:51:16.175253 9993 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 03:51:16.175283 9993 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 03:51:16.176719 9993 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38969
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785988004637995-12279-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785988004637995-12279-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39641
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785988004637995-12279-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39641
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785988004637995-12279-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9993
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785988004637995-12279-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:51:16.177014 9993 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:51:16.177248 9993 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:51:16.182236 9998 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:51:16.182281 9999 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:51:16.182264 10001 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:51:16.182390 9993 server_base.cc:1034] running on GCE node
I20260806 03:51:16.182900 9993 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:51:16.183234 9993 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:51:16.184386 9993 hybrid_clock.cc:648] HybridClock initialized: now 1785988276184368 us; error 30 us; skew 500 ppm
Aug 06 03:51:16 dist-test-slave-6ntk krb5kdc[9974](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988276, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:51:16.187736 9993 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 03:51:16.188326 9993 webserver.cc:460] Webserver started at http://127.0.0.1:46435/ using document root <none> and password file <none>
I20260806 03:51:16.188649 9993 fs_manager.cc:359] Metadata directory not provided
I20260806 03:51:16.188737 9993 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:51:16.188911 9993 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:51:16.190485 9993 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785988004637995-12279-0/minicluster-data/master-0/data/instance:
uuid: "2adb7ee5798c405899e0869119cd09e0"
format_stamp: "Formatted at 2026-08-06 03:51:16 on dist-test-slave-6ntk"
server_key: "8776b7991c92921bc71404903d3cc08e"
server_key_iv: "10a6787c6c18dd88e20d0b35b196e8d0"
server_key_version: "encryptionkey@0"
I20260806 03:51:16.190968 9993 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785988004637995-12279-0/minicluster-data/master-0/wal/instance:
uuid: "2adb7ee5798c405899e0869119cd09e0"
format_stamp: "Formatted at 2026-08-06 03:51:16 on dist-test-slave-6ntk"
server_key: "8776b7991c92921bc71404903d3cc08e"
server_key_iv: "10a6787c6c18dd88e20d0b35b196e8d0"
server_key_version: "encryptionkey@0"
I20260806 03:51:16.192571 9993 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 03:51:16.193390 10010 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:51:16.193599 9993 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 03:51:16.193681 9993 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785988004637995-12279-0/minicluster-data/master-0/data,/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785988004637995-12279-0/minicluster-data/master-0/wal
uuid: "2adb7ee5798c405899e0869119cd09e0"
format_stamp: "Formatted at 2026-08-06 03:51:16 on dist-test-slave-6ntk"
server_key: "8776b7991c92921bc71404903d3cc08e"
server_key_iv: "10a6787c6c18dd88e20d0b35b196e8d0"
server_key_version: "encryptionkey@0"
I20260806 03:51:16.193740 9993 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785988004637995-12279-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785988004637995-12279-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785988004637995-12279-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:51:16.198302 9993 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:51:16.201170 9993 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:51:16.201372 9993 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:51:16.205648 9993 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39641
I20260806 03:51:16.205681 10077 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39641 every 8 connection(s)
I20260806 03:51:16.206115 9993 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785988004637995-12279-0/minicluster-data/master-0/data/info.pb
I20260806 03:51:16.207165 10078 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:51:16.209471 10078 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2adb7ee5798c405899e0869119cd09e0: Bootstrap starting.
I20260806 03:51:16.210143 10078 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2adb7ee5798c405899e0869119cd09e0: Neither blocks nor log segments found. Creating new log.
I20260806 03:51:16.210450 10078 log.cc:824] T 00000000000000000000000000000000 P 2adb7ee5798c405899e0869119cd09e0: Log is configured to *not* fsync() on all Append() calls
I20260806 03:51:16.211154 10078 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2adb7ee5798c405899e0869119cd09e0: No bootstrap required, opened a new log
I20260806 03:51:16.212939 10078 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2adb7ee5798c405899e0869119cd09e0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2adb7ee5798c405899e0869119cd09e0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39641 } }
I20260806 03:51:16.213079 10078 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2adb7ee5798c405899e0869119cd09e0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:51:16.213102 10078 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2adb7ee5798c405899e0869119cd09e0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2adb7ee5798c405899e0869119cd09e0, State: Initialized, Role: FOLLOWER
I20260806 03:51:16.213267 10078 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2adb7ee5798c405899e0869119cd09e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2adb7ee5798c405899e0869119cd09e0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39641 } }
I20260806 03:51:16.213327 10078 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2adb7ee5798c405899e0869119cd09e0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 03:51:16.213387 10078 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2adb7ee5798c405899e0869119cd09e0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 03:51:16.213450 10078 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2adb7ee5798c405899e0869119cd09e0 [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:51:16.214082 10078 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2adb7ee5798c405899e0869119cd09e0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2adb7ee5798c405899e0869119cd09e0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39641 } }
I20260806 03:51:16.214198 10078 leader_election.cc:302] T 00000000000000000000000000000000 P 2adb7ee5798c405899e0869119cd09e0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2adb7ee5798c405899e0869119cd09e0; no voters:
I20260806 03:51:16.214380 10078 leader_election.cc:288] T 00000000000000000000000000000000 P 2adb7ee5798c405899e0869119cd09e0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 03:51:16.214461 10082 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2adb7ee5798c405899e0869119cd09e0 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 03:51:16.214650 10078 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2adb7ee5798c405899e0869119cd09e0 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 03:51:16.214633 10082 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2adb7ee5798c405899e0869119cd09e0 [term 1 LEADER]: Becoming Leader. State: Replica: 2adb7ee5798c405899e0869119cd09e0, State: Running, Role: LEADER
I20260806 03:51:16.214852 10082 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2adb7ee5798c405899e0869119cd09e0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2adb7ee5798c405899e0869119cd09e0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39641 } }
I20260806 03:51:16.215330 12279 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskToVWRe/build/release/bin/kudu as pid 9993
I20260806 03:51:16.215476 12279 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785988004637995-12279-0/minicluster-data/master-0/wal/instance
I20260806 03:51:16.215703 12279 external_mini_cluster.cc:1428] Setting key ad5c9db336b8b831ed3e2eba1716eaa4
I20260806 03:51:16.215693 10083 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2adb7ee5798c405899e0869119cd09e0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2adb7ee5798c405899e0869119cd09e0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2adb7ee5798c405899e0869119cd09e0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39641 } } }
I20260806 03:51:16.215796 10083 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2adb7ee5798c405899e0869119cd09e0 [sys.catalog]: This master's current role is: LEADER
I20260806 03:51:16.216181 10084 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2adb7ee5798c405899e0869119cd09e0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2adb7ee5798c405899e0869119cd09e0. Latest consensus state: current_term: 1 leader_uuid: "2adb7ee5798c405899e0869119cd09e0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2adb7ee5798c405899e0869119cd09e0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39641 } } }
I20260806 03:51:16.216614 10084 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2adb7ee5798c405899e0869119cd09e0 [sys.catalog]: This master's current role is: LEADER
I20260806 03:51:16.216181 10096 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 03:51:16.217159 10096 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 03:51:16.218168 10102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:16.216482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.193:41089 (local address 127.0.0.1:39641)
0806 03:51:16.216640 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0806 03:51:16.216650 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 03:51:16.216686 (+ 36us) server_negotiation.cc:407] Connection header received
0806 03:51:16.216805 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:51:16.216811 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:51:16.216834 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:51:16.216997 (+ 163us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:51:16.217681 (+ 684us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:51:16.217685 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:51:16.217685 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 03:51:16.217690 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:51:16.217732 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 03:51:16.217748 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:51:16.217820 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0806 03:51:16.217957 (+ 137us) server_negotiation.cc:299] Negotiation successful
0806 03:51:16.218005 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108,"thread_start_us":77,"threads_started":1}
I20260806 03:51:16.219117 12279 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 03:51:16.219143 10096 catalog_manager.cc:1347] Generated new cluster ID: 0eb2725c548840f7b930a64f34a3054c
I20260806 03:51:16.219189 10096 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 03:51:16.226992 10102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:16.220765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.193:55443 (local address 127.0.0.1:39641)
0806 03:51:16.225950 (+ 5185us) server_negotiation.cc:206] Beginning negotiation
0806 03:51:16.225952 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:51:16.225961 (+ 9us) server_negotiation.cc:407] Connection header received
0806 03:51:16.225982 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:51:16.225983 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:51:16.225988 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:51:16.226045 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:51:16.226746 (+ 701us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:51:16.226747 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:51:16.226748 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 03:51:16.226752 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:51:16.226768 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 03:51:16.226772 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:51:16.226830 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0806 03:51:16.226903 (+ 73us) server_negotiation.cc:299] Negotiation successful
0806 03:51:16.226916 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5154}
I20260806 03:51:16.257225 10096 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 03:51:16.257458 10096 catalog_manager.cc:1504] Loading token signing keys...
I20260806 03:51:16.260390 10096 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2adb7ee5798c405899e0869119cd09e0: Generated new TSK 0
I20260806 03:51:16.260623 10096 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 03:51:16.270666 12279 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskToVWRe/build/release/bin/kudu with pid 9993
2026-08-06T03:51:16Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1560 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785988004637995-12279-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 03:51:16 dist-test-slave-6ntk krb5kdc[10115](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 03:51:16 dist-test-slave-6ntk krb5kdc[10115](info): set up 2 sockets
Aug 06 03:51:16 dist-test-slave-6ntk krb5kdc[10115](info): commencing operation
krb5kdc: starting...
W20260806 03:51:17.492980 12279 mini_kdc.cc:121] Time spent starting KDC: real 1.195s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785988004637995-12279-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785988004637995-12279-0/krb5kdc/test-user.keytab.
Aug 06 03:51:17 dist-test-slave-6ntk krb5kdc[10115](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988277, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T03:51:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T03:51:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.253.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.253.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.253.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.254.keytab.
Entry for principal kudu/127.11.253.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.254.keytab.
I20260806 03:51:17.627221 12279 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskToVWRe/build/release/bin/kudu
/tmp/dist-test-taskToVWRe/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785988004637995-12279-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785988004637995-12279-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785988004637995-12279-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785988004637995-12279-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.253.254:46255
--webserver_interface=127.11.253.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.254.keytab
--principal=kudu/127.11.253.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.253.212:41419
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.253.254:46255
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785988004637995-12279-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785988004637995-12279-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785988004637995-12279-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:51:17.711959 10133 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:51:17.712232 10133 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:51:17.712273 10133 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:51:17.713522 10133 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 03:51:17.713608 10133 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:51:17.713632 10133 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:51:17.713653 10133 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 03:51:17.713685 10133 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 03:51:17.715240 10133 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.253.212:41419
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785988004637995-12279-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785988004637995-12279-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.253.254:46255
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.254.keytab
--principal=kudu/127.11.253.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.253.254:46255
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785988004637995-12279-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.253.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10133
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785988004637995-12279-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:51:17.715533 10133 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:51:17.715781 10133 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:51:17.720970 10141 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:51:17.720970 10144 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:51:17.720997 10140 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:51:17.721434 10133 server_base.cc:1034] running on GCE node
I20260806 03:51:17.721680 10133 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:51:17.722023 10133 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:51:17.723178 10133 hybrid_clock.cc:648] HybridClock initialized: now 1785988277723157 us; error 41 us; skew 500 ppm
Aug 06 03:51:17 dist-test-slave-6ntk krb5kdc[10115](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988277, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.253.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:51:17.728832 10133 init.cc:377] Logged in from keytab as kudu/127.11.253.254@KRBTEST.COM (short username kudu)
I20260806 03:51:17.729334 10133 webserver.cc:460] Webserver started at http://127.11.253.254:32841/ using document root <none> and password file <none>
I20260806 03:51:17.729616 10133 fs_manager.cc:359] Metadata directory not provided
I20260806 03:51:17.729667 10133 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:51:17.729840 10133 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:51:17.731065 10133 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785988004637995-12279-0/minicluster-data/master-0/data/instance:
uuid: "9e5906f0e4474bc2a9041d3f4eb13a92"
format_stamp: "Formatted at 2026-08-06 03:51:17 on dist-test-slave-6ntk"
server_key: "8fa7750cbad6eae8ba64e2e9f9cd5a50"
server_key_iv: "1d6285d223f77f60abfd0e09c7c0554a"
server_key_version: "encryptionkey@0"
I20260806 03:51:17.731451 10133 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785988004637995-12279-0/minicluster-data/master-0/wal/instance:
uuid: "9e5906f0e4474bc2a9041d3f4eb13a92"
format_stamp: "Formatted at 2026-08-06 03:51:17 on dist-test-slave-6ntk"
server_key: "8fa7750cbad6eae8ba64e2e9f9cd5a50"
server_key_iv: "1d6285d223f77f60abfd0e09c7c0554a"
server_key_version: "encryptionkey@0"
I20260806 03:51:17.732893 10133 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 03:51:17.733752 10154 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:51:17.734031 10133 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260806 03:51:17.734140 10133 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785988004637995-12279-0/minicluster-data/master-0/data,/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785988004637995-12279-0/minicluster-data/master-0/wal
uuid: "9e5906f0e4474bc2a9041d3f4eb13a92"
format_stamp: "Formatted at 2026-08-06 03:51:17 on dist-test-slave-6ntk"
server_key: "8fa7750cbad6eae8ba64e2e9f9cd5a50"
server_key_iv: "1d6285d223f77f60abfd0e09c7c0554a"
server_key_version: "encryptionkey@0"
I20260806 03:51:17.734212 10133 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785988004637995-12279-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785988004637995-12279-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785988004637995-12279-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:51:17.743935 10133 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:51:17.745954 10133 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:51:17.746125 10133 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:51:17.750434 10133 rpc_server.cc:307] RPC server started. Bound to: 127.11.253.254:46255
I20260806 03:51:17.750519 10220 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.253.254:46255 every 8 connection(s)
I20260806 03:51:17.750880 10133 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785988004637995-12279-0/minicluster-data/master-0/data/info.pb
I20260806 03:51:17.751873 10221 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:51:17.753293 12279 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskToVWRe/build/release/bin/kudu as pid 10133
I20260806 03:51:17.753443 12279 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785988004637995-12279-0/minicluster-data/master-0/wal/instance
I20260806 03:51:17.753633 12279 external_mini_cluster.cc:1428] Setting key a58d5f2690fcc0c2904ec8c3d3e7707a
I20260806 03:51:17.755120 10221 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9e5906f0e4474bc2a9041d3f4eb13a92: Bootstrap starting.
I20260806 03:51:17.755869 10221 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9e5906f0e4474bc2a9041d3f4eb13a92: Neither blocks nor log segments found. Creating new log.
I20260806 03:51:17.756253 10221 log.cc:824] T 00000000000000000000000000000000 P 9e5906f0e4474bc2a9041d3f4eb13a92: Log is configured to *not* fsync() on all Append() calls
I20260806 03:51:17.757079 10221 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9e5906f0e4474bc2a9041d3f4eb13a92: No bootstrap required, opened a new log
I20260806 03:51:17.759886 10221 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9e5906f0e4474bc2a9041d3f4eb13a92 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e5906f0e4474bc2a9041d3f4eb13a92" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 46255 } }
I20260806 03:51:17.760102 10221 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9e5906f0e4474bc2a9041d3f4eb13a92 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:51:17.760154 10221 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9e5906f0e4474bc2a9041d3f4eb13a92 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e5906f0e4474bc2a9041d3f4eb13a92, State: Initialized, Role: FOLLOWER
I20260806 03:51:17.760329 10221 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9e5906f0e4474bc2a9041d3f4eb13a92 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e5906f0e4474bc2a9041d3f4eb13a92" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 46255 } }
I20260806 03:51:17.760437 10221 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9e5906f0e4474bc2a9041d3f4eb13a92 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 03:51:17.760481 10221 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9e5906f0e4474bc2a9041d3f4eb13a92 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 03:51:17.760537 10221 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9e5906f0e4474bc2a9041d3f4eb13a92 [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:51:17.761303 10221 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9e5906f0e4474bc2a9041d3f4eb13a92 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e5906f0e4474bc2a9041d3f4eb13a92" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 46255 } }
Aug 06 03:51:17 dist-test-slave-6ntk krb5kdc[10115](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988277, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.253.254@KRBTEST.COM
I20260806 03:51:17.761409 10221 leader_election.cc:302] T 00000000000000000000000000000000 P 9e5906f0e4474bc2a9041d3f4eb13a92 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9e5906f0e4474bc2a9041d3f4eb13a92; no voters:
I20260806 03:51:17.761628 10221 leader_election.cc:288] T 00000000000000000000000000000000 P 9e5906f0e4474bc2a9041d3f4eb13a92 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 03:51:17.761662 10227 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9e5906f0e4474bc2a9041d3f4eb13a92 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 03:51:17.761796 10227 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9e5906f0e4474bc2a9041d3f4eb13a92 [term 1 LEADER]: Becoming Leader. State: Replica: 9e5906f0e4474bc2a9041d3f4eb13a92, State: Running, Role: LEADER
I20260806 03:51:17.761979 10227 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9e5906f0e4474bc2a9041d3f4eb13a92 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e5906f0e4474bc2a9041d3f4eb13a92" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 46255 } }
I20260806 03:51:17.761984 10221 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9e5906f0e4474bc2a9041d3f4eb13a92 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 03:51:17.762662 10230 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9e5906f0e4474bc2a9041d3f4eb13a92 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9e5906f0e4474bc2a9041d3f4eb13a92. Latest consensus state: current_term: 1 leader_uuid: "9e5906f0e4474bc2a9041d3f4eb13a92" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e5906f0e4474bc2a9041d3f4eb13a92" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 46255 } } }
I20260806 03:51:17.762851 10230 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9e5906f0e4474bc2a9041d3f4eb13a92 [sys.catalog]: This master's current role is: LEADER
I20260806 03:51:17.762636 10228 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9e5906f0e4474bc2a9041d3f4eb13a92 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9e5906f0e4474bc2a9041d3f4eb13a92" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e5906f0e4474bc2a9041d3f4eb13a92" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 46255 } } }
I20260806 03:51:17.763160 10228 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9e5906f0e4474bc2a9041d3f4eb13a92 [sys.catalog]: This master's current role is: LEADER
I20260806 03:51:17.763336 10238 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 03:51:17.763994 10238 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 03:51:17.766345 10225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:17.754210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54125 (local address 127.11.253.254:46255)
0806 03:51:17.754419 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0806 03:51:17.754430 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 03:51:17.754472 (+ 42us) server_negotiation.cc:407] Connection header received
0806 03:51:17.755050 (+ 578us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:51:17.755060 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:51:17.755130 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:51:17.755314 (+ 184us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:51:17.756158 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:17.757158 (+ 1000us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:17.758423 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:17.758863 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:17.758961 (+ 98us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:17.762081 (+ 3120us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:51:17.762086 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:51:17.762086 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:51:17.762099 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:51:17.764298 (+ 2199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:51:17.764798 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:51:17.764801 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:51:17.764802 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:51:17.764900 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:51:17.765339 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:51:17.765340 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:51:17.765340 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:51:17.765616 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:51:17.765717 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0806 03:51:17.766064 (+ 347us) server_negotiation.cc:299] Negotiation successful
0806 03:51:17.766171 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":89,"threads_started":1}
I20260806 03:51:17.767439 12279 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 03:51:17.770467 10238 catalog_manager.cc:1347] Generated new cluster ID: 25bc8b793e904a36a6132f86cb2acdfd
I20260806 03:51:17.770530 10238 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 03:51:17.775594 10225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:17.768845 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45687 (local address 127.11.253.254:46255)
0806 03:51:17.768880 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0806 03:51:17.768882 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:51:17.769006 (+ 124us) server_negotiation.cc:407] Connection header received
0806 03:51:17.769081 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:51:17.769082 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:51:17.769091 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:51:17.769140 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:51:17.769686 (+ 546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:17.770751 (+ 1065us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:17.771471 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:17.771708 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:17.771783 (+ 75us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:17.772992 (+ 1209us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:51:17.772993 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:51:17.772993 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:51:17.773000 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:51:17.774404 (+ 1404us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:51:17.774775 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:51:17.774777 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:51:17.774777 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:51:17.774833 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:51:17.775071 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:51:17.775071 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:51:17.775072 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:51:17.775187 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:51:17.775311 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0806 03:51:17.775435 (+ 124us) server_negotiation.cc:299] Negotiation successful
0806 03:51:17.775498 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260806 03:51:17.788656 10238 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 03:51:17.789208 10238 catalog_manager.cc:1504] Loading token signing keys...
I20260806 03:51:17.797001 10238 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9e5906f0e4474bc2a9041d3f4eb13a92: Generated new TSK 0
I20260806 03:51:17.797200 10238 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 03:51:17.801664 12279 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskToVWRe/build/release/bin/kudu with pid 10133
W20260806 03:51:17.813373 10113 connection.cc:535] client connection to 127.11.253.254:46255 recv error: Network error: failed to read from TLS socket (remote: 127.11.253.254:46255): Connection reset by peer (error 104)
2026-08-06T03:51:17Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1529 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785988004637995-12279-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 03:51:17 dist-test-slave-6ntk krb5kdc[10260](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 03:51:17 dist-test-slave-6ntk krb5kdc[10260](info): set up 2 sockets
Aug 06 03:51:17 dist-test-slave-6ntk krb5kdc[10260](info): commencing operation
krb5kdc: starting...
W20260806 03:51:19.021524 12279 mini_kdc.cc:121] Time spent starting KDC: real 1.194s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785988004637995-12279-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785988004637995-12279-0/krb5kdc/test-user.keytab.
Aug 06 03:51:19 dist-test-slave-6ntk krb5kdc[10260](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988279, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T03:51:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T03:51:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.253.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.253.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.253.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.254.keytab.
Entry for principal kudu/127.11.253.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.254.keytab.
I20260806 03:51:19.156422 12279 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskToVWRe/build/release/bin/kudu
/tmp/dist-test-taskToVWRe/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785988004637995-12279-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785988004637995-12279-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785988004637995-12279-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785988004637995-12279-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.253.254:34207
--webserver_interface=127.11.253.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.254.keytab
--principal=kudu/127.11.253.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.253.212:41381
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.253.254:34207
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785988004637995-12279-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785988004637995-12279-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785988004637995-12279-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:51:19.241160 10275 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:51:19.241377 10275 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:51:19.241400 10275 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:51:19.242595 10275 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 03:51:19.242641 10275 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:51:19.242656 10275 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:51:19.242669 10275 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 03:51:19.242681 10275 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 03:51:19.244201 10275 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.253.212:41381
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785988004637995-12279-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785988004637995-12279-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.253.254:34207
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.254.keytab
--principal=kudu/127.11.253.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.253.254:34207
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785988004637995-12279-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.253.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10275
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785988004637995-12279-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:51:19.244470 10275 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:51:19.244720 10275 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:51:19.250037 10285 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:51:19.250039 10282 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:51:19.250038 10281 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:51:19.250339 10275 server_base.cc:1034] running on GCE node
I20260806 03:51:19.250600 10275 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:51:19.250906 10275 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:51:19.252085 10275 hybrid_clock.cc:648] HybridClock initialized: now 1785988279252072 us; error 48 us; skew 500 ppm
Aug 06 03:51:19 dist-test-slave-6ntk krb5kdc[10260](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988279, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.253.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:51:19.254822 10275 init.cc:377] Logged in from keytab as kudu/127.11.253.254@KRBTEST.COM (short username kudu)
I20260806 03:51:19.255333 10275 webserver.cc:460] Webserver started at http://127.11.253.254:33509/ using document root <none> and password file <none>
I20260806 03:51:19.255577 10275 fs_manager.cc:359] Metadata directory not provided
I20260806 03:51:19.255650 10275 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:51:19.255820 10275 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:51:19.256955 10275 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785988004637995-12279-0/minicluster-data/master-0/data/instance:
uuid: "2f564887dc2b4bf5bdaea129d8cfb0e9"
format_stamp: "Formatted at 2026-08-06 03:51:19 on dist-test-slave-6ntk"
server_key: "fd5df7aaa81ca6d59804432760d0cc1a"
server_key_iv: "ba2f99e3ccea389d8b1806dd9dd760fe"
server_key_version: "encryptionkey@0"
I20260806 03:51:19.257361 10275 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785988004637995-12279-0/minicluster-data/master-0/wal/instance:
uuid: "2f564887dc2b4bf5bdaea129d8cfb0e9"
format_stamp: "Formatted at 2026-08-06 03:51:19 on dist-test-slave-6ntk"
server_key: "fd5df7aaa81ca6d59804432760d0cc1a"
server_key_iv: "ba2f99e3ccea389d8b1806dd9dd760fe"
server_key_version: "encryptionkey@0"
I20260806 03:51:19.258921 10275 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 03:51:19.259864 10292 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:51:19.260035 10275 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 03:51:19.260143 10275 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785988004637995-12279-0/minicluster-data/master-0/data,/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785988004637995-12279-0/minicluster-data/master-0/wal
uuid: "2f564887dc2b4bf5bdaea129d8cfb0e9"
format_stamp: "Formatted at 2026-08-06 03:51:19 on dist-test-slave-6ntk"
server_key: "fd5df7aaa81ca6d59804432760d0cc1a"
server_key_iv: "ba2f99e3ccea389d8b1806dd9dd760fe"
server_key_version: "encryptionkey@0"
I20260806 03:51:19.260226 10275 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785988004637995-12279-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785988004637995-12279-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785988004637995-12279-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:51:19.287436 10275 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:51:19.289788 10275 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:51:19.289997 10275 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:51:19.294350 10275 rpc_server.cc:307] RPC server started. Bound to: 127.11.253.254:34207
I20260806 03:51:19.294471 10366 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.253.254:34207 every 8 connection(s)
I20260806 03:51:19.294860 10275 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785988004637995-12279-0/minicluster-data/master-0/data/info.pb
I20260806 03:51:19.295825 10367 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:51:19.298345 10367 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2f564887dc2b4bf5bdaea129d8cfb0e9: Bootstrap starting.
I20260806 03:51:19.298951 10367 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2f564887dc2b4bf5bdaea129d8cfb0e9: Neither blocks nor log segments found. Creating new log.
I20260806 03:51:19.299264 10367 log.cc:824] T 00000000000000000000000000000000 P 2f564887dc2b4bf5bdaea129d8cfb0e9: Log is configured to *not* fsync() on all Append() calls
I20260806 03:51:19.300037 10367 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2f564887dc2b4bf5bdaea129d8cfb0e9: No bootstrap required, opened a new log
I20260806 03:51:19.301714 10367 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2f564887dc2b4bf5bdaea129d8cfb0e9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f564887dc2b4bf5bdaea129d8cfb0e9" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 34207 } }
I20260806 03:51:19.301857 10367 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2f564887dc2b4bf5bdaea129d8cfb0e9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:51:19.301970 10367 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2f564887dc2b4bf5bdaea129d8cfb0e9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f564887dc2b4bf5bdaea129d8cfb0e9, State: Initialized, Role: FOLLOWER
I20260806 03:51:19.302138 12279 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskToVWRe/build/release/bin/kudu as pid 10275
I20260806 03:51:19.302150 10367 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2f564887dc2b4bf5bdaea129d8cfb0e9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f564887dc2b4bf5bdaea129d8cfb0e9" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 34207 } }
I20260806 03:51:19.302248 10367 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2f564887dc2b4bf5bdaea129d8cfb0e9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 03:51:19.302310 10367 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2f564887dc2b4bf5bdaea129d8cfb0e9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 03:51:19.302316 12279 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785988004637995-12279-0/minicluster-data/master-0/wal/instance
I20260806 03:51:19.302363 10367 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2f564887dc2b4bf5bdaea129d8cfb0e9 [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:51:19.302522 12279 external_mini_cluster.cc:1428] Setting key d777dd8082368cffb22e690d4afae630
I20260806 03:51:19.303584 10367 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2f564887dc2b4bf5bdaea129d8cfb0e9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f564887dc2b4bf5bdaea129d8cfb0e9" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 34207 } }
I20260806 03:51:19.303706 10367 leader_election.cc:302] T 00000000000000000000000000000000 P 2f564887dc2b4bf5bdaea129d8cfb0e9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2f564887dc2b4bf5bdaea129d8cfb0e9; no voters:
I20260806 03:51:19.303941 10367 leader_election.cc:288] T 00000000000000000000000000000000 P 2f564887dc2b4bf5bdaea129d8cfb0e9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 03:51:19.303970 10373 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2f564887dc2b4bf5bdaea129d8cfb0e9 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 03:51:19.304088 10373 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2f564887dc2b4bf5bdaea129d8cfb0e9 [term 1 LEADER]: Becoming Leader. State: Replica: 2f564887dc2b4bf5bdaea129d8cfb0e9, State: Running, Role: LEADER
I20260806 03:51:19.304224 10367 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2f564887dc2b4bf5bdaea129d8cfb0e9 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 03:51:19.304246 10373 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2f564887dc2b4bf5bdaea129d8cfb0e9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f564887dc2b4bf5bdaea129d8cfb0e9" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 34207 } }
I20260806 03:51:19.304730 10376 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2f564887dc2b4bf5bdaea129d8cfb0e9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2f564887dc2b4bf5bdaea129d8cfb0e9. Latest consensus state: current_term: 1 leader_uuid: "2f564887dc2b4bf5bdaea129d8cfb0e9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f564887dc2b4bf5bdaea129d8cfb0e9" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 34207 } } }
I20260806 03:51:19.304840 10376 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2f564887dc2b4bf5bdaea129d8cfb0e9 [sys.catalog]: This master's current role is: LEADER
I20260806 03:51:19.305081 10374 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2f564887dc2b4bf5bdaea129d8cfb0e9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2f564887dc2b4bf5bdaea129d8cfb0e9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f564887dc2b4bf5bdaea129d8cfb0e9" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 34207 } } }
I20260806 03:51:19.305204 10374 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2f564887dc2b4bf5bdaea129d8cfb0e9 [sys.catalog]: This master's current role is: LEADER
I20260806 03:51:19.305377 10383 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 03:51:19.306030 10383 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 03:51:19.307761 10383 catalog_manager.cc:1347] Generated new cluster ID: b52b9754d62c4c47a49fbef5ad245344
I20260806 03:51:19.307849 10383 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 03:51:19.308648 10372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:19.303079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53989 (local address 127.11.253.254:34207)
0806 03:51:19.303327 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0806 03:51:19.303337 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 03:51:19.303374 (+ 37us) server_negotiation.cc:407] Connection header received
0806 03:51:19.303493 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:51:19.303500 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:51:19.303558 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:51:19.303716 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:51:19.303987 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:19.305370 (+ 1383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:19.306550 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:19.307034 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:19.307175 (+ 141us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:19.307931 (+ 756us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:51:19.307935 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:51:19.307935 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 03:51:19.308159 (+ 224us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:51:19.308196 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 03:51:19.308210 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:51:19.308290 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0806 03:51:19.308417 (+ 127us) server_negotiation.cc:299] Negotiation successful
0806 03:51:19.308445 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":183,"thread_start_us":116,"threads_started":1}
I20260806 03:51:19.309652 12279 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 03:51:19.315042 10372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:19.311335 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48607 (local address 127.11.253.254:34207)
0806 03:51:19.311394 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0806 03:51:19.311396 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:51:19.311605 (+ 209us) server_negotiation.cc:407] Connection header received
0806 03:51:19.311645 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:51:19.311646 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:51:19.311657 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:51:19.311723 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:51:19.311822 (+ 99us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:19.312550 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:19.313529 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:19.313900 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:19.313972 (+ 72us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:19.314785 (+ 813us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:51:19.314786 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:51:19.314787 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 03:51:19.314792 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:51:19.314808 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 03:51:19.314812 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:51:19.314878 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0806 03:51:19.314951 (+ 73us) server_negotiation.cc:299] Negotiation successful
0806 03:51:19.314963 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260806 03:51:19.318437 10383 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 03:51:19.318944 10383 catalog_manager.cc:1504] Loading token signing keys...
I20260806 03:51:19.321161 10383 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2f564887dc2b4bf5bdaea129d8cfb0e9: Generated new TSK 0
I20260806 03:51:19.321339 10383 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 03:51:19.325690 12279 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskToVWRe/build/release/bin/kudu with pid 10275
W20260806 03:51:19.339681 10258 connection.cc:535] client connection to 127.11.253.254:34207 recv error: Network error: failed to read from TLS socket (remote: 127.11.253.254:34207): Connection reset by peer (error 104)
2026-08-06T03:51:19Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1525 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785988004637995-12279-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 03:51:19 dist-test-slave-6ntk krb5kdc[10404](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 03:51:19 dist-test-slave-6ntk krb5kdc[10404](info): set up 2 sockets
Aug 06 03:51:19 dist-test-slave-6ntk krb5kdc[10404](info): commencing operation
krb5kdc: starting...
W20260806 03:51:20.550904 12279 mini_kdc.cc:121] Time spent starting KDC: real 1.198s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785988004637995-12279-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785988004637995-12279-0/krb5kdc/test-user.keytab.
Aug 06 03:51:20 dist-test-slave-6ntk krb5kdc[10404](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988280, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T03:51:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T03:51:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.253.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.253.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.253.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.254.keytab.
Entry for principal kudu/127.11.253.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.254.keytab.
I20260806 03:51:20.683069 12279 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskToVWRe/build/release/bin/kudu
/tmp/dist-test-taskToVWRe/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785988004637995-12279-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785988004637995-12279-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785988004637995-12279-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785988004637995-12279-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.253.254:37891
--webserver_interface=127.11.253.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.254.keytab
--principal=kudu/127.11.253.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.253.212:33027
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.253.254:37891
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785988004637995-12279-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785988004637995-12279-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785988004637995-12279-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:51:20.767455 10424 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:51:20.767721 10424 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:51:20.767761 10424 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:51:20.768980 10424 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 03:51:20.769063 10424 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:51:20.769088 10424 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:51:20.769107 10424 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 03:51:20.769141 10424 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 03:51:20.770726 10424 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.253.212:33027
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785988004637995-12279-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785988004637995-12279-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.253.254:37891
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.254.keytab
--principal=kudu/127.11.253.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.253.254:37891
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785988004637995-12279-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.253.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10424
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785988004637995-12279-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:51:20.771039 10424 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:51:20.771296 10424 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:51:20.776755 10432 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:51:20.776690 10435 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:51:20.776727 10433 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:51:20.776723 10424 server_base.cc:1034] running on GCE node
I20260806 03:51:20.777204 10424 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:51:20.777505 10424 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:51:20.778673 10424 hybrid_clock.cc:648] HybridClock initialized: now 1785988280778660 us; error 49 us; skew 500 ppm
Aug 06 03:51:20 dist-test-slave-6ntk krb5kdc[10404](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988280, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.253.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:51:20.781423 10424 init.cc:377] Logged in from keytab as kudu/127.11.253.254@KRBTEST.COM (short username kudu)
I20260806 03:51:20.781966 10424 webserver.cc:460] Webserver started at http://127.11.253.254:32775/ using document root <none> and password file <none>
I20260806 03:51:20.782198 10424 fs_manager.cc:359] Metadata directory not provided
I20260806 03:51:20.782269 10424 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:51:20.782428 10424 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:51:20.783557 10424 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785988004637995-12279-0/minicluster-data/master-0/data/instance:
uuid: "501ed6b82556475a883a91b8c9422e0d"
format_stamp: "Formatted at 2026-08-06 03:51:20 on dist-test-slave-6ntk"
server_key: "841ff7bfbce0c10f85b50e9954ba9da7"
server_key_iv: "1a290a2f33aaaed841ab5ab5afbfe882"
server_key_version: "encryptionkey@0"
I20260806 03:51:20.783948 10424 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785988004637995-12279-0/minicluster-data/master-0/wal/instance:
uuid: "501ed6b82556475a883a91b8c9422e0d"
format_stamp: "Formatted at 2026-08-06 03:51:20 on dist-test-slave-6ntk"
server_key: "841ff7bfbce0c10f85b50e9954ba9da7"
server_key_iv: "1a290a2f33aaaed841ab5ab5afbfe882"
server_key_version: "encryptionkey@0"
I20260806 03:51:20.785322 10424 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 03:51:20.786173 10444 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:51:20.786383 10424 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 03:51:20.786492 10424 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785988004637995-12279-0/minicluster-data/master-0/data,/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785988004637995-12279-0/minicluster-data/master-0/wal
uuid: "501ed6b82556475a883a91b8c9422e0d"
format_stamp: "Formatted at 2026-08-06 03:51:20 on dist-test-slave-6ntk"
server_key: "841ff7bfbce0c10f85b50e9954ba9da7"
server_key_iv: "1a290a2f33aaaed841ab5ab5afbfe882"
server_key_version: "encryptionkey@0"
I20260806 03:51:20.786571 10424 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785988004637995-12279-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785988004637995-12279-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785988004637995-12279-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:51:20.789999 10424 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:51:20.791923 10424 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:51:20.792091 10424 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:51:20.796335 10424 rpc_server.cc:307] RPC server started. Bound to: 127.11.253.254:37891
I20260806 03:51:20.796391 10514 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.253.254:37891 every 8 connection(s)
I20260806 03:51:20.796734 10424 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785988004637995-12279-0/minicluster-data/master-0/data/info.pb
I20260806 03:51:20.797775 10515 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:51:20.798700 12279 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskToVWRe/build/release/bin/kudu as pid 10424
I20260806 03:51:20.798847 12279 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785988004637995-12279-0/minicluster-data/master-0/wal/instance
I20260806 03:51:20.799046 12279 external_mini_cluster.cc:1428] Setting key ae35dd9596caeb25af9f24b37e90b78d
I20260806 03:51:20.801290 10515 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 501ed6b82556475a883a91b8c9422e0d: Bootstrap starting.
I20260806 03:51:20.801730 10518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:20.799657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36573 (local address 127.11.253.254:37891)
0806 03:51:20.799870 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0806 03:51:20.799878 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 03:51:20.799904 (+ 26us) server_negotiation.cc:407] Connection header received
0806 03:51:20.800321 (+ 417us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:51:20.800327 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:51:20.800361 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:51:20.800483 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:51:20.801183 (+ 700us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:51:20.801186 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:51:20.801187 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 03:51:20.801190 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:51:20.801287 (+ 97us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 03:51:20.801299 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:51:20.801358 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0806 03:51:20.801460 (+ 102us) server_negotiation.cc:299] Negotiation successful
0806 03:51:20.801530 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":155,"thread_start_us":120,"threads_started":1}
I20260806 03:51:20.802043 10515 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 501ed6b82556475a883a91b8c9422e0d: Neither blocks nor log segments found. Creating new log.
I20260806 03:51:20.802423 10515 log.cc:824] T 00000000000000000000000000000000 P 501ed6b82556475a883a91b8c9422e0d: Log is configured to *not* fsync() on all Append() calls
I20260806 03:51:20.802829 12279 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 03:51:20.803279 10515 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 501ed6b82556475a883a91b8c9422e0d: No bootstrap required, opened a new log
I20260806 03:51:20.805202 10515 raft_consensus.cc:348] T 00000000000000000000000000000000 P 501ed6b82556475a883a91b8c9422e0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "501ed6b82556475a883a91b8c9422e0d" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 37891 } }
I20260806 03:51:20.805373 10515 raft_consensus.cc:374] T 00000000000000000000000000000000 P 501ed6b82556475a883a91b8c9422e0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:51:20.805421 10515 raft_consensus.cc:729] T 00000000000000000000000000000000 P 501ed6b82556475a883a91b8c9422e0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 501ed6b82556475a883a91b8c9422e0d, State: Initialized, Role: FOLLOWER
I20260806 03:51:20.805562 10515 consensus_queue.cc:260] T 00000000000000000000000000000000 P 501ed6b82556475a883a91b8c9422e0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "501ed6b82556475a883a91b8c9422e0d" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 37891 } }
I20260806 03:51:20.805660 10515 raft_consensus.cc:388] T 00000000000000000000000000000000 P 501ed6b82556475a883a91b8c9422e0d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 03:51:20.805706 10515 raft_consensus.cc:482] T 00000000000000000000000000000000 P 501ed6b82556475a883a91b8c9422e0d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 03:51:20.805766 10515 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 501ed6b82556475a883a91b8c9422e0d [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:51:20.806581 10515 raft_consensus.cc:504] T 00000000000000000000000000000000 P 501ed6b82556475a883a91b8c9422e0d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "501ed6b82556475a883a91b8c9422e0d" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 37891 } }
I20260806 03:51:20.806732 10515 leader_election.cc:302] T 00000000000000000000000000000000 P 501ed6b82556475a883a91b8c9422e0d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 501ed6b82556475a883a91b8c9422e0d; no voters:
I20260806 03:51:20.806952 10515 leader_election.cc:288] T 00000000000000000000000000000000 P 501ed6b82556475a883a91b8c9422e0d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 03:51:20.806952 10518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:20.804320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37561 (local address 127.11.253.254:37891)
0806 03:51:20.805917 (+ 1597us) server_negotiation.cc:206] Beginning negotiation
0806 03:51:20.805919 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:51:20.805941 (+ 22us) server_negotiation.cc:407] Connection header received
0806 03:51:20.805963 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:51:20.805963 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:51:20.805968 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:51:20.806034 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:51:20.806687 (+ 653us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:51:20.806688 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:51:20.806688 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 03:51:20.806700 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:51:20.806715 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 03:51:20.806719 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:51:20.806781 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0806 03:51:20.806861 (+ 80us) server_negotiation.cc:299] Negotiation successful
0806 03:51:20.806875 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1575}
I20260806 03:51:20.807004 10527 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 501ed6b82556475a883a91b8c9422e0d [term 1 FOLLOWER]: Leader election won for term 1
I20260806 03:51:20.807382 10527 raft_consensus.cc:686] T 00000000000000000000000000000000 P 501ed6b82556475a883a91b8c9422e0d [term 1 LEADER]: Becoming Leader. State: Replica: 501ed6b82556475a883a91b8c9422e0d, State: Running, Role: LEADER
I20260806 03:51:20.807489 10527 consensus_queue.cc:237] T 00000000000000000000000000000000 P 501ed6b82556475a883a91b8c9422e0d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "501ed6b82556475a883a91b8c9422e0d" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 37891 } }
I20260806 03:51:20.807538 10515 sys_catalog.cc:564] T 00000000000000000000000000000000 P 501ed6b82556475a883a91b8c9422e0d [sys.catalog]: configured and running, proceeding with master startup.
I20260806 03:51:20.807817 10527 sys_catalog.cc:455] T 00000000000000000000000000000000 P 501ed6b82556475a883a91b8c9422e0d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 501ed6b82556475a883a91b8c9422e0d. Latest consensus state: current_term: 1 leader_uuid: "501ed6b82556475a883a91b8c9422e0d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "501ed6b82556475a883a91b8c9422e0d" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 37891 } } }
I20260806 03:51:20.807911 10527 sys_catalog.cc:458] T 00000000000000000000000000000000 P 501ed6b82556475a883a91b8c9422e0d [sys.catalog]: This master's current role is: LEADER
I20260806 03:51:20.807829 10529 sys_catalog.cc:455] T 00000000000000000000000000000000 P 501ed6b82556475a883a91b8c9422e0d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "501ed6b82556475a883a91b8c9422e0d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "501ed6b82556475a883a91b8c9422e0d" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 37891 } } }
I20260806 03:51:20.808029 10529 sys_catalog.cc:458] T 00000000000000000000000000000000 P 501ed6b82556475a883a91b8c9422e0d [sys.catalog]: This master's current role is: LEADER
I20260806 03:51:20.808714 10532 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 03:51:20.809259 10532 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 03:51:20.810997 10532 catalog_manager.cc:1347] Generated new cluster ID: 433e9d43bc6f44e8a45ec1f747223ff0
I20260806 03:51:20.811053 10532 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 03:51:20.821740 10532 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 03:51:20.821946 10532 catalog_manager.cc:1504] Loading token signing keys...
I20260806 03:51:20.833478 10532 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 501ed6b82556475a883a91b8c9422e0d: Generated new TSK 0
I20260806 03:51:20.833734 10532 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 03:51:20.837973 12279 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskToVWRe/build/release/bin/kudu with pid 10424
2026-08-06T03:51:20Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1512 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785988004637995-12279-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 03:51:20 dist-test-slave-6ntk krb5kdc[10552](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 03:51:20 dist-test-slave-6ntk krb5kdc[10552](info): set up 2 sockets
Aug 06 03:51:20 dist-test-slave-6ntk krb5kdc[10552](info): commencing operation
krb5kdc: starting...
W20260806 03:51:22.053326 12279 mini_kdc.cc:121] Time spent starting KDC: real 1.188s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785988004637995-12279-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785988004637995-12279-0/krb5kdc/test-user.keytab.
Aug 06 03:51:22 dist-test-slave-6ntk krb5kdc[10552](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988282, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T03:51:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T03:51:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.253.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.253.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.253.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.254.keytab.
Entry for principal kudu/127.11.253.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.254.keytab.
I20260806 03:51:22.185745 12279 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskToVWRe/build/release/bin/kudu
/tmp/dist-test-taskToVWRe/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785988004637995-12279-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785988004637995-12279-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785988004637995-12279-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785988004637995-12279-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.253.254:43709
--webserver_interface=127.11.253.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.254.keytab
--principal=kudu/127.11.253.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.253.212:46391
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.253.254:43709
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785988004637995-12279-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785988004637995-12279-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785988004637995-12279-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:51:22.274813 10567 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:51:22.275054 10567 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:51:22.275079 10567 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:51:22.276374 10567 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 03:51:22.276428 10567 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:51:22.276443 10567 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:51:22.276455 10567 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 03:51:22.276468 10567 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 03:51:22.278177 10567 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.253.212:46391
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785988004637995-12279-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785988004637995-12279-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.253.254:43709
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.254.keytab
--principal=kudu/127.11.253.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.253.254:43709
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785988004637995-12279-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.253.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10567
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785988004637995-12279-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:51:22.278469 10567 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:51:22.278724 10567 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:51:22.284065 10580 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:51:22.284161 10567 server_base.cc:1034] running on GCE node
W20260806 03:51:22.284026 10578 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:51:22.284037 10577 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:51:22.284650 10567 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:51:22.284973 10567 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:51:22.286132 10567 hybrid_clock.cc:648] HybridClock initialized: now 1785988282286112 us; error 40 us; skew 500 ppm
Aug 06 03:51:22 dist-test-slave-6ntk krb5kdc[10552](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988282, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.253.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:51:22.288805 10567 init.cc:377] Logged in from keytab as kudu/127.11.253.254@KRBTEST.COM (short username kudu)
I20260806 03:51:22.289330 10567 webserver.cc:460] Webserver started at http://127.11.253.254:35117/ using document root <none> and password file <none>
I20260806 03:51:22.289619 10567 fs_manager.cc:359] Metadata directory not provided
I20260806 03:51:22.289696 10567 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:51:22.289855 10567 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:51:22.290982 10567 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785988004637995-12279-0/minicluster-data/master-0/data/instance:
uuid: "07965accf46e4945b0537d0140dd9f90"
format_stamp: "Formatted at 2026-08-06 03:51:22 on dist-test-slave-6ntk"
server_key: "b91f83e3a36ffb2e067b4f60d1cd4a07"
server_key_iv: "25ca09a118b318cb0e7fe8927eb01049"
server_key_version: "encryptionkey@0"
I20260806 03:51:22.291365 10567 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785988004637995-12279-0/minicluster-data/master-0/wal/instance:
uuid: "07965accf46e4945b0537d0140dd9f90"
format_stamp: "Formatted at 2026-08-06 03:51:22 on dist-test-slave-6ntk"
server_key: "b91f83e3a36ffb2e067b4f60d1cd4a07"
server_key_iv: "25ca09a118b318cb0e7fe8927eb01049"
server_key_version: "encryptionkey@0"
I20260806 03:51:22.292814 10567 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 03:51:22.293598 10591 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:51:22.293790 10567 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 03:51:22.293927 10567 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785988004637995-12279-0/minicluster-data/master-0/data,/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785988004637995-12279-0/minicluster-data/master-0/wal
uuid: "07965accf46e4945b0537d0140dd9f90"
format_stamp: "Formatted at 2026-08-06 03:51:22 on dist-test-slave-6ntk"
server_key: "b91f83e3a36ffb2e067b4f60d1cd4a07"
server_key_iv: "25ca09a118b318cb0e7fe8927eb01049"
server_key_version: "encryptionkey@0"
I20260806 03:51:22.294016 10567 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785988004637995-12279-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785988004637995-12279-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785988004637995-12279-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:51:22.316211 10567 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:51:22.318557 10567 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:51:22.318740 10567 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:51:22.323379 10567 rpc_server.cc:307] RPC server started. Bound to: 127.11.253.254:43709
I20260806 03:51:22.323421 10657 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.253.254:43709 every 8 connection(s)
I20260806 03:51:22.323817 10567 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785988004637995-12279-0/minicluster-data/master-0/data/info.pb
I20260806 03:51:22.324770 10658 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:51:22.327256 10658 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 07965accf46e4945b0537d0140dd9f90: Bootstrap starting.
I20260806 03:51:22.327900 10658 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 07965accf46e4945b0537d0140dd9f90: Neither blocks nor log segments found. Creating new log.
I20260806 03:51:22.328205 10658 log.cc:824] T 00000000000000000000000000000000 P 07965accf46e4945b0537d0140dd9f90: Log is configured to *not* fsync() on all Append() calls
I20260806 03:51:22.329010 10658 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 07965accf46e4945b0537d0140dd9f90: No bootstrap required, opened a new log
I20260806 03:51:22.330796 10658 raft_consensus.cc:348] T 00000000000000000000000000000000 P 07965accf46e4945b0537d0140dd9f90 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "07965accf46e4945b0537d0140dd9f90" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 43709 } }
I20260806 03:51:22.330971 10658 raft_consensus.cc:374] T 00000000000000000000000000000000 P 07965accf46e4945b0537d0140dd9f90 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:51:22.331027 10658 raft_consensus.cc:729] T 00000000000000000000000000000000 P 07965accf46e4945b0537d0140dd9f90 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07965accf46e4945b0537d0140dd9f90, State: Initialized, Role: FOLLOWER
I20260806 03:51:22.331177 10658 consensus_queue.cc:260] T 00000000000000000000000000000000 P 07965accf46e4945b0537d0140dd9f90 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "07965accf46e4945b0537d0140dd9f90" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 43709 } }
I20260806 03:51:22.331276 10658 raft_consensus.cc:388] T 00000000000000000000000000000000 P 07965accf46e4945b0537d0140dd9f90 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 03:51:22.331319 10658 raft_consensus.cc:482] T 00000000000000000000000000000000 P 07965accf46e4945b0537d0140dd9f90 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 03:51:22.331380 10658 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 07965accf46e4945b0537d0140dd9f90 [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:51:22.332309 10658 raft_consensus.cc:504] T 00000000000000000000000000000000 P 07965accf46e4945b0537d0140dd9f90 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "07965accf46e4945b0537d0140dd9f90" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 43709 } }
I20260806 03:51:22.332448 10658 leader_election.cc:302] T 00000000000000000000000000000000 P 07965accf46e4945b0537d0140dd9f90 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 07965accf46e4945b0537d0140dd9f90; no voters:
I20260806 03:51:22.332660 10658 leader_election.cc:288] T 00000000000000000000000000000000 P 07965accf46e4945b0537d0140dd9f90 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 03:51:22.332705 10663 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 07965accf46e4945b0537d0140dd9f90 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 03:51:22.332902 10663 raft_consensus.cc:686] T 00000000000000000000000000000000 P 07965accf46e4945b0537d0140dd9f90 [term 1 LEADER]: Becoming Leader. State: Replica: 07965accf46e4945b0537d0140dd9f90, State: Running, Role: LEADER
I20260806 03:51:22.332948 10658 sys_catalog.cc:564] T 00000000000000000000000000000000 P 07965accf46e4945b0537d0140dd9f90 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 03:51:22.333070 10663 consensus_queue.cc:237] T 00000000000000000000000000000000 P 07965accf46e4945b0537d0140dd9f90 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "07965accf46e4945b0537d0140dd9f90" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 43709 } }
I20260806 03:51:22.333343 12279 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskToVWRe/build/release/bin/kudu as pid 10567
I20260806 03:51:22.333446 12279 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785988004637995-12279-0/minicluster-data/master-0/wal/instance
I20260806 03:51:22.333667 12279 external_mini_cluster.cc:1428] Setting key 9335a9c98945d1042c51654afbe7602d
I20260806 03:51:22.334179 10664 sys_catalog.cc:455] T 00000000000000000000000000000000 P 07965accf46e4945b0537d0140dd9f90 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "07965accf46e4945b0537d0140dd9f90" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "07965accf46e4945b0537d0140dd9f90" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 43709 } } }
I20260806 03:51:22.334188 10667 sys_catalog.cc:455] T 00000000000000000000000000000000 P 07965accf46e4945b0537d0140dd9f90 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 07965accf46e4945b0537d0140dd9f90. Latest consensus state: current_term: 1 leader_uuid: "07965accf46e4945b0537d0140dd9f90" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "07965accf46e4945b0537d0140dd9f90" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 43709 } } }
I20260806 03:51:22.334326 10664 sys_catalog.cc:458] T 00000000000000000000000000000000 P 07965accf46e4945b0537d0140dd9f90 [sys.catalog]: This master's current role is: LEADER
I20260806 03:51:22.334332 10667 sys_catalog.cc:458] T 00000000000000000000000000000000 P 07965accf46e4945b0537d0140dd9f90 [sys.catalog]: This master's current role is: LEADER
I20260806 03:51:22.334539 10682 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 03:51:22.335204 10682 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 03:51:22.336911 10682 catalog_manager.cc:1347] Generated new cluster ID: 1ea86a29a9a64072b0620f79ef346aba
I20260806 03:51:22.336973 10682 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 03:51:22 dist-test-slave-6ntk krb5kdc[10552](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988282, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.253.254@KRBTEST.COM
I20260806 03:51:22.345117 10685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:22.334682 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53503 (local address 127.11.253.254:43709)
0806 03:51:22.334936 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0806 03:51:22.334948 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 03:51:22.334991 (+ 43us) server_negotiation.cc:407] Connection header received
0806 03:51:22.335116 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:51:22.335123 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:51:22.335184 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:51:22.335355 (+ 171us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:51:22.335997 (+ 642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:22.337253 (+ 1256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:22.338140 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:22.338555 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:22.338690 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:22.341131 (+ 2441us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:51:22.341136 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:51:22.341136 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:51:22.341148 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:51:22.343445 (+ 2297us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:51:22.343848 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:51:22.343851 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:51:22.343852 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:51:22.343970 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:51:22.344186 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:51:22.344187 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:51:22.344188 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:51:22.344471 (+ 283us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:51:22.344651 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0806 03:51:22.344754 (+ 103us) server_negotiation.cc:299] Negotiation successful
0806 03:51:22.344911 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":191,"thread_start_us":80,"threads_started":1}
I20260806 03:51:22.346186 12279 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 03:51:22.346863 10682 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 03:51:22.347576 10682 catalog_manager.cc:1504] Loading token signing keys...
I20260806 03:51:22.351626 10682 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 07965accf46e4945b0537d0140dd9f90: Generated new TSK 0
I20260806 03:51:22.351929 10682 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 03:51:22.363219 10685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:22.347822 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42685 (local address 127.11.253.254:43709)
0806 03:51:22.347862 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0806 03:51:22.347865 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 03:51:22.353941 (+ 6076us) server_negotiation.cc:407] Connection header received
0806 03:51:22.353980 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:51:22.353981 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:51:22.354000 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:51:22.354086 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:51:22.355144 (+ 1058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:22.356137 (+ 993us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:22.357515 (+ 1378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:22.357906 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:22.357972 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:22.359797 (+ 1825us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:51:22.359798 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:51:22.359799 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:51:22.359807 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:51:22.361782 (+ 1975us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:51:22.362309 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:51:22.362311 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:51:22.362311 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:51:22.362376 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:51:22.362621 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:51:22.362622 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:51:22.362622 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:51:22.362761 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:51:22.362882 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0806 03:51:22.363028 (+ 146us) server_negotiation.cc:299] Negotiation successful
0806 03:51:22.363114 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 03:51:22.365577 12279 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskToVWRe/build/release/bin/kudu with pid 10567
2026-08-06T03:51:22Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1526 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785988004637995-12279-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 03:51:22 dist-test-slave-6ntk krb5kdc[10700](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 03:51:22 dist-test-slave-6ntk krb5kdc[10700](info): set up 2 sockets
Aug 06 03:51:22 dist-test-slave-6ntk krb5kdc[10700](info): commencing operation
krb5kdc: starting...
W20260806 03:51:23.595997 12279 mini_kdc.cc:121] Time spent starting KDC: real 1.204s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785988004637995-12279-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785988004637995-12279-0/krb5kdc/test-user.keytab.
Aug 06 03:51:23 dist-test-slave-6ntk krb5kdc[10700](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988283, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T03:51:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T03:51:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.253.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.253.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.253.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.254.keytab.
Entry for principal kudu/127.11.253.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.254.keytab.
I20260806 03:51:23.728993 12279 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskToVWRe/build/release/bin/kudu
/tmp/dist-test-taskToVWRe/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785988004637995-12279-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785988004637995-12279-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785988004637995-12279-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785988004637995-12279-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.253.254:41135
--webserver_interface=127.11.253.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.254.keytab
--principal=kudu/127.11.253.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.253.212:34363
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.253.254:41135
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785988004637995-12279-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785988004637995-12279-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785988004637995-12279-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:51:23.814814 10718 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:51:23.815085 10718 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:51:23.815124 10718 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:51:23.816311 10718 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 03:51:23.816394 10718 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:51:23.816418 10718 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:51:23.816440 10718 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 03:51:23.816469 10718 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 03:51:23.817965 10718 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.253.212:34363
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785988004637995-12279-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785988004637995-12279-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.253.254:41135
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.254.keytab
--principal=kudu/127.11.253.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.253.254:41135
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785988004637995-12279-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.253.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10718
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785988004637995-12279-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:51:23.818265 10718 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:51:23.818506 10718 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:51:23.823516 10724 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:51:23.823498 10729 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:51:23.823702 10718 server_base.cc:1034] running on GCE node
W20260806 03:51:23.823493 10725 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:51:23.824059 10718 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:51:23.824420 10718 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:51:23.825590 10718 hybrid_clock.cc:648] HybridClock initialized: now 1785988283825568 us; error 44 us; skew 500 ppm
Aug 06 03:51:23 dist-test-slave-6ntk krb5kdc[10700](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988283, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.253.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:51:23.828261 10718 init.cc:377] Logged in from keytab as kudu/127.11.253.254@KRBTEST.COM (short username kudu)
I20260806 03:51:23.828727 10718 webserver.cc:460] Webserver started at http://127.11.253.254:33459/ using document root <none> and password file <none>
I20260806 03:51:23.829010 10718 fs_manager.cc:359] Metadata directory not provided
I20260806 03:51:23.829080 10718 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:51:23.829245 10718 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:51:23.830408 10718 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785988004637995-12279-0/minicluster-data/master-0/data/instance:
uuid: "e6cc46ef32fe43d287cfd082bb4f8aa0"
format_stamp: "Formatted at 2026-08-06 03:51:23 on dist-test-slave-6ntk"
server_key: "00bc9e630f158262bbffefe8edf739c4"
server_key_iv: "8d1882e2732af66a0f08bcd5b0719565"
server_key_version: "encryptionkey@0"
I20260806 03:51:23.830852 10718 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785988004637995-12279-0/minicluster-data/master-0/wal/instance:
uuid: "e6cc46ef32fe43d287cfd082bb4f8aa0"
format_stamp: "Formatted at 2026-08-06 03:51:23 on dist-test-slave-6ntk"
server_key: "00bc9e630f158262bbffefe8edf739c4"
server_key_iv: "8d1882e2732af66a0f08bcd5b0719565"
server_key_version: "encryptionkey@0"
I20260806 03:51:23.832247 10718 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 03:51:23.833006 10739 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:51:23.833313 10718 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 03:51:23.833426 10718 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785988004637995-12279-0/minicluster-data/master-0/data,/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785988004637995-12279-0/minicluster-data/master-0/wal
uuid: "e6cc46ef32fe43d287cfd082bb4f8aa0"
format_stamp: "Formatted at 2026-08-06 03:51:23 on dist-test-slave-6ntk"
server_key: "00bc9e630f158262bbffefe8edf739c4"
server_key_iv: "8d1882e2732af66a0f08bcd5b0719565"
server_key_version: "encryptionkey@0"
I20260806 03:51:23.833504 10718 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785988004637995-12279-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785988004637995-12279-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785988004637995-12279-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:51:23.845089 10718 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:51:23.847129 10718 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:51:23.847306 10718 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:51:23.852007 10718 rpc_server.cc:307] RPC server started. Bound to: 127.11.253.254:41135
I20260806 03:51:23.852042 10805 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.253.254:41135 every 8 connection(s)
I20260806 03:51:23.852423 10718 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785988004637995-12279-0/minicluster-data/master-0/data/info.pb
I20260806 03:51:23.853367 10808 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:51:23.854878 12279 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskToVWRe/build/release/bin/kudu as pid 10718
I20260806 03:51:23.855001 12279 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785988004637995-12279-0/minicluster-data/master-0/wal/instance
I20260806 03:51:23.855195 12279 external_mini_cluster.cc:1428] Setting key 2a96b449253fa84891d5c5c2c7dd13ee
I20260806 03:51:23.856215 10808 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e6cc46ef32fe43d287cfd082bb4f8aa0: Bootstrap starting.
I20260806 03:51:23.856930 10808 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e6cc46ef32fe43d287cfd082bb4f8aa0: Neither blocks nor log segments found. Creating new log.
I20260806 03:51:23.857244 10808 log.cc:824] T 00000000000000000000000000000000 P e6cc46ef32fe43d287cfd082bb4f8aa0: Log is configured to *not* fsync() on all Append() calls
I20260806 03:51:23.858060 10808 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e6cc46ef32fe43d287cfd082bb4f8aa0: No bootstrap required, opened a new log
I20260806 03:51:23.860001 10808 raft_consensus.cc:348] T 00000000000000000000000000000000 P e6cc46ef32fe43d287cfd082bb4f8aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6cc46ef32fe43d287cfd082bb4f8aa0" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 41135 } }
I20260806 03:51:23.860172 10808 raft_consensus.cc:374] T 00000000000000000000000000000000 P e6cc46ef32fe43d287cfd082bb4f8aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:51:23.860220 10808 raft_consensus.cc:729] T 00000000000000000000000000000000 P e6cc46ef32fe43d287cfd082bb4f8aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6cc46ef32fe43d287cfd082bb4f8aa0, State: Initialized, Role: FOLLOWER
I20260806 03:51:23.860353 10808 consensus_queue.cc:260] T 00000000000000000000000000000000 P e6cc46ef32fe43d287cfd082bb4f8aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6cc46ef32fe43d287cfd082bb4f8aa0" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 41135 } }
I20260806 03:51:23.860435 10808 raft_consensus.cc:388] T 00000000000000000000000000000000 P e6cc46ef32fe43d287cfd082bb4f8aa0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 03:51:23.860481 10808 raft_consensus.cc:482] T 00000000000000000000000000000000 P e6cc46ef32fe43d287cfd082bb4f8aa0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 03:51:23.860532 10808 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e6cc46ef32fe43d287cfd082bb4f8aa0 [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:51:23.860854 10812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:23.855773 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42463 (local address 127.11.253.254:41135)
0806 03:51:23.855981 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0806 03:51:23.855990 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 03:51:23.856027 (+ 37us) server_negotiation.cc:407] Connection header received
0806 03:51:23.856604 (+ 577us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:51:23.856614 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:51:23.856688 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:51:23.856841 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:51:23.857047 (+ 206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:23.858090 (+ 1043us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:23.858871 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:23.859278 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:23.859411 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:23.860106 (+ 695us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:51:23.860110 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:51:23.860110 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 03:51:23.860377 (+ 267us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:51:23.860417 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 03:51:23.860429 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:51:23.860507 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0806 03:51:23.860610 (+ 103us) server_negotiation.cc:299] Negotiation successful
0806 03:51:23.860679 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":151,"thread_start_us":85,"threads_started":1}
I20260806 03:51:23.861258 10808 raft_consensus.cc:504] T 00000000000000000000000000000000 P e6cc46ef32fe43d287cfd082bb4f8aa0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6cc46ef32fe43d287cfd082bb4f8aa0" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 41135 } }
I20260806 03:51:23.861387 10808 leader_election.cc:302] T 00000000000000000000000000000000 P e6cc46ef32fe43d287cfd082bb4f8aa0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e6cc46ef32fe43d287cfd082bb4f8aa0; no voters:
I20260806 03:51:23.861549 10808 leader_election.cc:288] T 00000000000000000000000000000000 P e6cc46ef32fe43d287cfd082bb4f8aa0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 03:51:23.861573 10814 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e6cc46ef32fe43d287cfd082bb4f8aa0 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 03:51:23.861685 10814 raft_consensus.cc:686] T 00000000000000000000000000000000 P e6cc46ef32fe43d287cfd082bb4f8aa0 [term 1 LEADER]: Becoming Leader. State: Replica: e6cc46ef32fe43d287cfd082bb4f8aa0, State: Running, Role: LEADER
I20260806 03:51:23.861790 12279 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 03:51:23.861824 10808 sys_catalog.cc:564] T 00000000000000000000000000000000 P e6cc46ef32fe43d287cfd082bb4f8aa0 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 03:51:23.861819 10814 consensus_queue.cc:237] T 00000000000000000000000000000000 P e6cc46ef32fe43d287cfd082bb4f8aa0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6cc46ef32fe43d287cfd082bb4f8aa0" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 41135 } }
I20260806 03:51:23.862368 10815 sys_catalog.cc:455] T 00000000000000000000000000000000 P e6cc46ef32fe43d287cfd082bb4f8aa0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e6cc46ef32fe43d287cfd082bb4f8aa0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6cc46ef32fe43d287cfd082bb4f8aa0" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 41135 } } }
I20260806 03:51:23.862457 10815 sys_catalog.cc:458] T 00000000000000000000000000000000 P e6cc46ef32fe43d287cfd082bb4f8aa0 [sys.catalog]: This master's current role is: LEADER
I20260806 03:51:23.863569 10816 sys_catalog.cc:455] T 00000000000000000000000000000000 P e6cc46ef32fe43d287cfd082bb4f8aa0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e6cc46ef32fe43d287cfd082bb4f8aa0. Latest consensus state: current_term: 1 leader_uuid: "e6cc46ef32fe43d287cfd082bb4f8aa0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6cc46ef32fe43d287cfd082bb4f8aa0" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 41135 } } }
I20260806 03:51:23.863660 10816 sys_catalog.cc:458] T 00000000000000000000000000000000 P e6cc46ef32fe43d287cfd082bb4f8aa0 [sys.catalog]: This master's current role is: LEADER
W20260806 03:51:23.863857 10838 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e6cc46ef32fe43d287cfd082bb4f8aa0: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 03:51:23.863915 10838 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 03:51:23.863961 10821 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 03:51:23.864113 10821 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 03:51:23.865617 10821 catalog_manager.cc:1347] Generated new cluster ID: f9545a148f83414ca274fa210a29ade7
I20260806 03:51:23.865681 10821 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 03:51:23.868901 10812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:23.863461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41581 (local address 127.11.253.254:41135)
0806 03:51:23.863496 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0806 03:51:23.863498 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:51:23.866054 (+ 2556us) server_negotiation.cc:407] Connection header received
0806 03:51:23.866126 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:51:23.866127 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:51:23.866137 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:51:23.866193 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:51:23.866391 (+ 198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:23.866872 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:23.867607 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:23.867823 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:23.867896 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:23.868603 (+ 707us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:51:23.868605 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:51:23.868605 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 03:51:23.868610 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:51:23.868625 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 03:51:23.868629 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:51:23.868673 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0806 03:51:23.868748 (+ 75us) server_negotiation.cc:299] Negotiation successful
0806 03:51:23.868772 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260806 03:51:23.889673 10821 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 03:51:23.890518 10821 catalog_manager.cc:1504] Loading token signing keys...
I20260806 03:51:23.896006 10821 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e6cc46ef32fe43d287cfd082bb4f8aa0: Generated new TSK 0
I20260806 03:51:23.896219 10821 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 03:51:23.899430 12279 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskToVWRe/build/release/bin/kudu with pid 10718
2026-08-06T03:51:23Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1530 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785988004637995-12279-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 03:51:23 dist-test-slave-6ntk krb5kdc[10844](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 03:51:23 dist-test-slave-6ntk krb5kdc[10844](info): set up 2 sockets
Aug 06 03:51:23 dist-test-slave-6ntk krb5kdc[10844](info): commencing operation
krb5kdc: starting...
W20260806 03:51:25.173486 12279 mini_kdc.cc:121] Time spent starting KDC: real 1.251s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785988004637995-12279-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785988004637995-12279-0/krb5kdc/test-user.keytab.
Aug 06 03:51:25 dist-test-slave-6ntk krb5kdc[10844](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988285, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T03:51:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T03:51:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.253.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.253.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.253.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.254.keytab.
Entry for principal kudu/127.11.253.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.254.keytab.
I20260806 03:51:25.304427 12279 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskToVWRe/build/release/bin/kudu
/tmp/dist-test-taskToVWRe/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785988004637995-12279-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785988004637995-12279-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785988004637995-12279-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785988004637995-12279-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.253.254:45963
--webserver_interface=127.11.253.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.254.keytab
--principal=kudu/127.11.253.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.253.212:35635
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.253.254:45963
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785988004637995-12279-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785988004637995-12279-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785988004637995-12279-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:51:25.388341 10862 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:51:25.388554 10862 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:51:25.388577 10862 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:51:25.389962 10862 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 03:51:25.390025 10862 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:51:25.390041 10862 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:51:25.390054 10862 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 03:51:25.390067 10862 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 03:51:25.391749 10862 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.253.212:35635
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785988004637995-12279-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785988004637995-12279-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.253.254:45963
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.254.keytab
--principal=kudu/127.11.253.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.253.254:45963
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785988004637995-12279-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.253.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10862
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785988004637995-12279-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:51:25.391952 10862 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:51:25.392134 10862 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:51:25.397285 10868 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:51:25.397279 10870 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:51:25.397279 10867 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:51:25.397307 10862 server_base.cc:1034] running on GCE node
I20260806 03:51:25.397763 10862 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:51:25.398089 10862 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:51:25.399241 10862 hybrid_clock.cc:648] HybridClock initialized: now 1785988285399229 us; error 30 us; skew 500 ppm
Aug 06 03:51:25 dist-test-slave-6ntk krb5kdc[10844](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988285, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.253.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:51:25.401777 10862 init.cc:377] Logged in from keytab as kudu/127.11.253.254@KRBTEST.COM (short username kudu)
I20260806 03:51:25.402275 10862 webserver.cc:460] Webserver started at http://127.11.253.254:36177/ using document root <none> and password file <none>
I20260806 03:51:25.402464 10862 fs_manager.cc:359] Metadata directory not provided
I20260806 03:51:25.402506 10862 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:51:25.402596 10862 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:51:25.403650 10862 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785988004637995-12279-0/minicluster-data/master-0/data/instance:
uuid: "1248250341ec4a50929e8f92448ab4dd"
format_stamp: "Formatted at 2026-08-06 03:51:25 on dist-test-slave-6ntk"
server_key: "c6bfb2647d1238e12b3834e0744bd518"
server_key_iv: "8e255db72426e77bb4ca60513f555ac6"
server_key_version: "encryptionkey@0"
I20260806 03:51:25.403970 10862 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785988004637995-12279-0/minicluster-data/master-0/wal/instance:
uuid: "1248250341ec4a50929e8f92448ab4dd"
format_stamp: "Formatted at 2026-08-06 03:51:25 on dist-test-slave-6ntk"
server_key: "c6bfb2647d1238e12b3834e0744bd518"
server_key_iv: "8e255db72426e77bb4ca60513f555ac6"
server_key_version: "encryptionkey@0"
I20260806 03:51:25.405313 10862 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260806 03:51:25.406109 10880 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:51:25.406263 10862 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 03:51:25.406355 10862 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785988004637995-12279-0/minicluster-data/master-0/data,/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785988004637995-12279-0/minicluster-data/master-0/wal
uuid: "1248250341ec4a50929e8f92448ab4dd"
format_stamp: "Formatted at 2026-08-06 03:51:25 on dist-test-slave-6ntk"
server_key: "c6bfb2647d1238e12b3834e0744bd518"
server_key_iv: "8e255db72426e77bb4ca60513f555ac6"
server_key_version: "encryptionkey@0"
I20260806 03:51:25.406417 10862 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785988004637995-12279-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785988004637995-12279-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785988004637995-12279-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:51:25.409597 10862 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:51:25.411434 10862 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:51:25.411537 10862 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:51:25.415357 10862 rpc_server.cc:307] RPC server started. Bound to: 127.11.253.254:45963
I20260806 03:51:25.415390 10942 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.253.254:45963 every 8 connection(s)
I20260806 03:51:25.415714 10862 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785988004637995-12279-0/minicluster-data/master-0/data/info.pb
I20260806 03:51:25.416709 10945 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:51:25.419050 10945 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1248250341ec4a50929e8f92448ab4dd: Bootstrap starting.
I20260806 03:51:25.419641 10945 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1248250341ec4a50929e8f92448ab4dd: Neither blocks nor log segments found. Creating new log.
I20260806 03:51:25.419976 10945 log.cc:824] T 00000000000000000000000000000000 P 1248250341ec4a50929e8f92448ab4dd: Log is configured to *not* fsync() on all Append() calls
I20260806 03:51:25.419998 12279 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskToVWRe/build/release/bin/kudu as pid 10862
I20260806 03:51:25.420116 12279 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785988004637995-12279-0/minicluster-data/master-0/wal/instance
I20260806 03:51:25.420318 12279 external_mini_cluster.cc:1428] Setting key ec95984e573812cb01121eca5e61ff32
I20260806 03:51:25.420928 10945 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1248250341ec4a50929e8f92448ab4dd: No bootstrap required, opened a new log
I20260806 03:51:25.422775 10950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:25.420864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60005 (local address 127.11.253.254:45963)
0806 03:51:25.421046 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0806 03:51:25.421053 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 03:51:25.421078 (+ 25us) server_negotiation.cc:407] Connection header received
0806 03:51:25.421488 (+ 410us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:51:25.421493 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:51:25.421516 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:51:25.421625 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:51:25.422326 (+ 701us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:51:25.422329 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:51:25.422329 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 03:51:25.422333 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:51:25.422369 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 03:51:25.422380 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:51:25.422450 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0806 03:51:25.422523 (+ 73us) server_negotiation.cc:299] Negotiation successful
0806 03:51:25.422575 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":93,"threads_started":1}
I20260806 03:51:25.423846 12279 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 03:51:25.424017 10945 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1248250341ec4a50929e8f92448ab4dd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1248250341ec4a50929e8f92448ab4dd" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 45963 } }
I20260806 03:51:25.424172 10945 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1248250341ec4a50929e8f92448ab4dd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:51:25.424203 10945 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1248250341ec4a50929e8f92448ab4dd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1248250341ec4a50929e8f92448ab4dd, State: Initialized, Role: FOLLOWER
I20260806 03:51:25.424321 10945 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1248250341ec4a50929e8f92448ab4dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1248250341ec4a50929e8f92448ab4dd" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 45963 } }
I20260806 03:51:25.424415 10945 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1248250341ec4a50929e8f92448ab4dd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 03:51:25.424448 10945 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1248250341ec4a50929e8f92448ab4dd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 03:51:25.424499 10945 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1248250341ec4a50929e8f92448ab4dd [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:51:25.425272 10945 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1248250341ec4a50929e8f92448ab4dd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1248250341ec4a50929e8f92448ab4dd" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 45963 } }
I20260806 03:51:25.425410 10945 leader_election.cc:302] T 00000000000000000000000000000000 P 1248250341ec4a50929e8f92448ab4dd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1248250341ec4a50929e8f92448ab4dd; no voters:
I20260806 03:51:25.425563 10945 leader_election.cc:288] T 00000000000000000000000000000000 P 1248250341ec4a50929e8f92448ab4dd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 03:51:25.425590 10961 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1248250341ec4a50929e8f92448ab4dd [term 1 FOLLOWER]: Leader election won for term 1
I20260806 03:51:25.425711 10961 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1248250341ec4a50929e8f92448ab4dd [term 1 LEADER]: Becoming Leader. State: Replica: 1248250341ec4a50929e8f92448ab4dd, State: Running, Role: LEADER
I20260806 03:51:25.425803 10945 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1248250341ec4a50929e8f92448ab4dd [sys.catalog]: configured and running, proceeding with master startup.
I20260806 03:51:25.425837 10961 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1248250341ec4a50929e8f92448ab4dd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1248250341ec4a50929e8f92448ab4dd" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 45963 } }
I20260806 03:51:25.426293 10962 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1248250341ec4a50929e8f92448ab4dd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1248250341ec4a50929e8f92448ab4dd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1248250341ec4a50929e8f92448ab4dd" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 45963 } } }
I20260806 03:51:25.426344 10963 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1248250341ec4a50929e8f92448ab4dd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1248250341ec4a50929e8f92448ab4dd. Latest consensus state: current_term: 1 leader_uuid: "1248250341ec4a50929e8f92448ab4dd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1248250341ec4a50929e8f92448ab4dd" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 45963 } } }
I20260806 03:51:25.426391 10962 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1248250341ec4a50929e8f92448ab4dd [sys.catalog]: This master's current role is: LEADER
I20260806 03:51:25.426393 10963 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1248250341ec4a50929e8f92448ab4dd [sys.catalog]: This master's current role is: LEADER
I20260806 03:51:25.426851 10950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:25.425415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60023 (local address 127.11.253.254:45963)
0806 03:51:25.425451 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0806 03:51:25.425452 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 03:51:25.425661 (+ 209us) server_negotiation.cc:407] Connection header received
0806 03:51:25.425763 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:51:25.425764 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:51:25.425768 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:51:25.425798 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:51:25.426616 (+ 818us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:51:25.426617 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:51:25.426618 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 03:51:25.426622 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:51:25.426635 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 03:51:25.426639 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:51:25.426686 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0806 03:51:25.426766 (+ 80us) server_negotiation.cc:299] Negotiation successful
0806 03:51:25.426781 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
W20260806 03:51:25.427893 10982 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1248250341ec4a50929e8f92448ab4dd: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 03:51:25.428027 10982 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 03:51:25.428100 10968 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 03:51:25.428259 10968 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 03:51:25.429610 10968 catalog_manager.cc:1347] Generated new cluster ID: e2c2568d632240228521ec5c7611fcea
I20260806 03:51:25.429661 10968 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 03:51:25.446645 10968 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 03:51:25.446853 10968 catalog_manager.cc:1504] Loading token signing keys...
I20260806 03:51:25.450835 10968 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1248250341ec4a50929e8f92448ab4dd: Generated new TSK 0
I20260806 03:51:25.451046 10968 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 03:51:25.453856 12279 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskToVWRe/build/release/bin/kudu with pid 10862
2026-08-06T03:51:25Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1557 ms)
[----------] 18 tests from AuthTokenIssuingTest (27814 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785988004637995-12279-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 03:51:25 dist-test-slave-6ntk krb5kdc[10991](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 03:51:25 dist-test-slave-6ntk krb5kdc[10991](info): set up 2 sockets
Aug 06 03:51:25 dist-test-slave-6ntk krb5kdc[10991](info): commencing operation
krb5kdc: starting...
W20260806 03:51:26.673453 12279 mini_kdc.cc:121] Time spent starting KDC: real 1.193s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785988004637995-12279-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785988004637995-12279-0/krb5kdc/test-user.keytab.
Aug 06 03:51:26 dist-test-slave-6ntk krb5kdc[10991](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988286, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T03:51:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T03:51:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.253.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.253.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.253.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.254.keytab.
Entry for principal kudu/127.11.253.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.254.keytab.
I20260806 03:51:26.808957 12279 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskToVWRe/build/release/bin/kudu
/tmp/dist-test-taskToVWRe/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785988004637995-12279-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785988004637995-12279-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785988004637995-12279-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785988004637995-12279-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.253.254:37231
--webserver_interface=127.11.253.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.254.keytab
--principal=kudu/127.11.253.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.253.212:40457
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.253.254:37231,127.11.253.253:36387,127.11.253.252:35087
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785988004637995-12279-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785988004637995-12279-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785988004637995-12279-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:51:26.894621 11009 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 03:51:26.894853 11009 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:51:26.894872 11009 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:51:26.894893 11009 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:51:26.896011 11009 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 03:51:26.896045 11009 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:51:26.896060 11009 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:51:26.896072 11009 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 03:51:26.896085 11009 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 03:51:26.897519 11009 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.253.212:40457
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785988004637995-12279-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785988004637995-12279-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.253.254:37231,127.11.253.253:36387,127.11.253.252:35087
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.254.keytab
--principal=kudu/127.11.253.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.253.254:37231
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785988004637995-12279-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.253.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11009
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785988004637995-12279-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:51:26.897720 11009 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:51:26.897930 11009 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:51:26.902971 11017 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:51:26.902925 11016 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:51:26.902910 11019 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:51:26.902989 11009 server_base.cc:1034] running on GCE node
I20260806 03:51:26.903354 11009 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:51:26.903689 11009 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:51:26.904860 11009 hybrid_clock.cc:648] HybridClock initialized: now 1785988286904837 us; error 42 us; skew 500 ppm
Aug 06 03:51:26 dist-test-slave-6ntk krb5kdc[10991](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988286, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.253.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:51:26.907637 11009 init.cc:377] Logged in from keytab as kudu/127.11.253.254@KRBTEST.COM (short username kudu)
I20260806 03:51:26.908147 11009 webserver.cc:460] Webserver started at http://127.11.253.254:39143/ using document root <none> and password file <none>
I20260806 03:51:26.908408 11009 fs_manager.cc:359] Metadata directory not provided
I20260806 03:51:26.908458 11009 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:51:26.908646 11009 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:51:26.910007 11009 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785988004637995-12279-0/minicluster-data/master-0/data/instance:
uuid: "3b37dc1a270849ebbed01fe4798c3c01"
format_stamp: "Formatted at 2026-08-06 03:51:26 on dist-test-slave-6ntk"
server_key: "b68d1163b136ebef3be32b0556550569"
server_key_iv: "c87bf2b9b5e49af26e89f996da569c2e"
server_key_version: "encryptionkey@0"
I20260806 03:51:26.910441 11009 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785988004637995-12279-0/minicluster-data/master-0/wal/instance:
uuid: "3b37dc1a270849ebbed01fe4798c3c01"
format_stamp: "Formatted at 2026-08-06 03:51:26 on dist-test-slave-6ntk"
server_key: "b68d1163b136ebef3be32b0556550569"
server_key_iv: "c87bf2b9b5e49af26e89f996da569c2e"
server_key_version: "encryptionkey@0"
I20260806 03:51:26.912078 11009 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 03:51:26.912954 11027 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:51:26.913161 11009 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 03:51:26.913271 11009 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785988004637995-12279-0/minicluster-data/master-0/data,/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785988004637995-12279-0/minicluster-data/master-0/wal
uuid: "3b37dc1a270849ebbed01fe4798c3c01"
format_stamp: "Formatted at 2026-08-06 03:51:26 on dist-test-slave-6ntk"
server_key: "b68d1163b136ebef3be32b0556550569"
server_key_iv: "c87bf2b9b5e49af26e89f996da569c2e"
server_key_version: "encryptionkey@0"
I20260806 03:51:26.913398 11009 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785988004637995-12279-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785988004637995-12279-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785988004637995-12279-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:51:26.922518 11009 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:51:26.924494 11009 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:51:26.924659 11009 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:51:26.929062 11009 rpc_server.cc:307] RPC server started. Bound to: 127.11.253.254:37231
I20260806 03:51:26.929117 11092 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.253.254:37231 every 8 connection(s)
I20260806 03:51:26.929512 11009 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785988004637995-12279-0/minicluster-data/master-0/data/info.pb
I20260806 03:51:26.930519 11093 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:51:26.932155 11093 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.253.254" port: 37231 } has no permanent_uuid. Determining permanent_uuid...
I20260806 03:51:26.934969 12279 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskToVWRe/build/release/bin/kudu as pid 11009
I20260806 03:51:26.935084 12279 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785988004637995-12279-0/minicluster-data/master-0/wal/instance
I20260806 03:51:26.935277 12279 external_mini_cluster.cc:1428] Setting key 9ca73b499b1cc1c511c9012f7c7f2f43
Aug 06 03:51:26 dist-test-slave-6ntk krb5kdc[10991](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988286, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.253.254@KRBTEST.COM for kudu/127.11.253.254@KRBTEST.COM
I20260806 03:51:26.942996 11096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:26.932851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:37231 (local address 127.0.0.1:52896)
0806 03:51:26.933022 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0806 03:51:26.933059 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0806 03:51:26.933519 (+ 460us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:51:26.933775 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:51:26.933778 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:51:26.934142 (+ 364us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:51:26.934298 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:51:26.934302 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:51:26.935447 (+ 1145us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:51:26.935449 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:51:26.936545 (+ 1096us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:51:26.936548 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:51:26.937036 (+ 488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:51:26.937038 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:51:26.937125 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:26.937739 (+ 614us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:51:26.937747 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:51:26.939212 (+ 1465us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:51:26.941496 (+ 2284us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:51:26.941503 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:51:26.941504 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:51:26.941704 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:51:26.942013 (+ 309us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:51:26.942014 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:51:26.942014 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:51:26.942097 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:51:26.942571 (+ 474us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:51:26.942575 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:51:26.942629 (+ 54us) client_negotiation.cc:769] Sending connection context
0806 03:51:26.942701 (+ 72us) client_negotiation.cc:240] Negotiation successful
0806 03:51:26.942827 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":72,"threads_started":1}
I20260806 03:51:26.943077 11097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:26.933010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52896 (local address 127.11.253.254:37231)
0806 03:51:26.933140 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0806 03:51:26.933144 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 03:51:26.933185 (+ 41us) server_negotiation.cc:407] Connection header received
0806 03:51:26.933647 (+ 462us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:51:26.933648 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:51:26.933665 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:51:26.933743 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:51:26.934389 (+ 646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:26.935331 (+ 942us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:26.936624 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:26.936946 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:26.937025 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:26.939319 (+ 2294us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:51:26.939327 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:51:26.939327 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:51:26.939340 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:51:26.941386 (+ 2046us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:51:26.941789 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:51:26.941793 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:51:26.941794 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:51:26.941940 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:51:26.942168 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:51:26.942169 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:51:26.942170 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:51:26.942495 (+ 325us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:51:26.942590 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0806 03:51:26.942767 (+ 177us) server_negotiation.cc:299] Negotiation successful
0806 03:51:26.942923 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":37,"server-negotiator.queue_time_us":97,"thread_start_us":69,"threads_started":1}
I20260806 03:51:26.943931 11093 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.253.253" port: 36387 } has no permanent_uuid. Determining permanent_uuid...
I20260806 03:51:26.944291 11096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:26.944168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.253:36387 (local address 127.0.0.1:51268)
0806 03:51:26.944217 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0806 03:51:26.944266 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.253.253:36387: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260806 03:51:26.944444 11031 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.11.253.253:36387: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.11.253.253@KRBTEST.COM; defaulting to no policy
I20260806 03:51:26.944893 11096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:26.944790 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.253:36387 (local address 127.0.0.1:51270)
0806 03:51:26.944847 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0806 03:51:26.944877 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.253.253:36387: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260806 03:51:26.944983 11093 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.253.253:36387: Network error: Client connection negotiation failed: client connection to 127.11.253.253:36387: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.253.253@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.253.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.253.keytab.
Entry for principal kudu/127.11.253.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.253.keytab.
I20260806 03:51:26.964794 12279 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskToVWRe/build/release/bin/kudu
/tmp/dist-test-taskToVWRe/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785988004637995-12279-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785988004637995-12279-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785988004637995-12279-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785988004637995-12279-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.253.253:36387
--webserver_interface=127.11.253.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.253.keytab
--principal=kudu/127.11.253.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.253.212:40457
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.253.254:37231,127.11.253.253:36387,127.11.253.252:35087
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785988004637995-12279-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785988004637995-12279-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785988004637995-12279-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 03:51:26.994225 11093 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.253.253" port: 36387 } attempt: 1
I20260806 03:51:26.994822 11096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:26.994654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.253:36387 (local address 127.0.0.1:51274)
0806 03:51:26.994738 (+ 84us) negotiation.cc:107] Waiting for socket to connect
0806 03:51:26.994785 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.253.253:36387: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260806 03:51:26.995270 11096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:26.995154 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.253:36387 (local address 127.0.0.1:51280)
0806 03:51:26.995208 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0806 03:51:26.995239 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.253.253:36387: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260806 03:51:26.995368 11093 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.253.253:36387: Network error: Client connection negotiation failed: client connection to 127.11.253.253:36387: connect: Connection refused (error 111)
W20260806 03:51:27.050091 11101 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 03:51:27.050334 11101 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:51:27.050366 11101 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:51:27.050406 11101 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:51:27.051550 11101 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 03:51:27.051620 11101 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:51:27.051643 11101 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:51:27.051666 11101 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 03:51:27.051693 11101 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 03:51:27.053162 11101 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.253.212:40457
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785988004637995-12279-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785988004637995-12279-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.11.253.254:37231,127.11.253.253:36387,127.11.253.252:35087
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.253.keytab
--principal=kudu/127.11.253.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.253.253:36387
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785988004637995-12279-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.11.253.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11101
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785988004637995-12279-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:51:27.053437 11101 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:51:27.053666 11101 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:51:27.058701 11109 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:51:27.058741 11111 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:51:27.058714 11107 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:51:27.058852 11101 server_base.cc:1034] running on GCE node
I20260806 03:51:27.059264 11101 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:51:27.059583 11101 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:51:27.060770 11101 hybrid_clock.cc:648] HybridClock initialized: now 1785988287060747 us; error 40 us; skew 500 ppm
Aug 06 03:51:27 dist-test-slave-6ntk krb5kdc[10991](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988287, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.253.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:51:27.063503 11093 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.253.253" port: 36387 } attempt: 2
I20260806 03:51:27.063690 11101 init.cc:377] Logged in from keytab as kudu/127.11.253.253@KRBTEST.COM (short username kudu)
I20260806 03:51:27.063943 11096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:27.063796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.253:36387 (local address 127.0.0.1:51288)
0806 03:51:27.063870 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0806 03:51:27.063916 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.253.253:36387: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260806 03:51:27.064282 11101 webserver.cc:460] Webserver started at http://127.11.253.253:40967/ using document root <none> and password file <none>
I20260806 03:51:27.064328 11096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:27.064186 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.253:36387 (local address 127.0.0.1:51300)
0806 03:51:27.064233 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0806 03:51:27.064259 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.253.253:36387: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260806 03:51:27.064441 11093 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.253.253:36387: Network error: Client connection negotiation failed: client connection to 127.11.253.253:36387: connect: Connection refused (error 111)
I20260806 03:51:27.064592 11101 fs_manager.cc:359] Metadata directory not provided
I20260806 03:51:27.064642 11101 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:51:27.064787 11101 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:51:27.065853 11101 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785988004637995-12279-0/minicluster-data/master-1/data/instance:
uuid: "b201733feb5f4a5c87ebf08707f39461"
format_stamp: "Formatted at 2026-08-06 03:51:27 on dist-test-slave-6ntk"
server_key: "92b59f26acd97cc21ad8fad2a2a0697d"
server_key_iv: "849753693a6f666ce073b434bf6643d2"
server_key_version: "encryptionkey@0"
I20260806 03:51:27.066365 11101 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785988004637995-12279-0/minicluster-data/master-1/wal/instance:
uuid: "b201733feb5f4a5c87ebf08707f39461"
format_stamp: "Formatted at 2026-08-06 03:51:27 on dist-test-slave-6ntk"
server_key: "92b59f26acd97cc21ad8fad2a2a0697d"
server_key_iv: "849753693a6f666ce073b434bf6643d2"
server_key_version: "encryptionkey@0"
I20260806 03:51:27.067709 11101 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260806 03:51:27.068519 11125 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:51:27.068720 11101 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 03:51:27.068815 11101 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785988004637995-12279-0/minicluster-data/master-1/data,/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785988004637995-12279-0/minicluster-data/master-1/wal
uuid: "b201733feb5f4a5c87ebf08707f39461"
format_stamp: "Formatted at 2026-08-06 03:51:27 on dist-test-slave-6ntk"
server_key: "92b59f26acd97cc21ad8fad2a2a0697d"
server_key_iv: "849753693a6f666ce073b434bf6643d2"
server_key_version: "encryptionkey@0"
I20260806 03:51:27.068874 11101 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785988004637995-12279-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785988004637995-12279-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785988004637995-12279-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:51:27.088845 11101 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:51:27.091420 11101 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:51:27.091542 11101 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:51:27.096737 11101 rpc_server.cc:307] RPC server started. Bound to: 127.11.253.253:36387
I20260806 03:51:27.096773 11188 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.253.253:36387 every 8 connection(s)
I20260806 03:51:27.097193 11101 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785988004637995-12279-0/minicluster-data/master-1/data/info.pb
I20260806 03:51:27.098274 11189 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:51:27.099895 11189 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.253.254" port: 37231 } has no permanent_uuid. Determining permanent_uuid...
I20260806 03:51:27.100656 12279 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskToVWRe/build/release/bin/kudu as pid 11101
I20260806 03:51:27.100769 12279 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785988004637995-12279-0/minicluster-data/master-1/wal/instance
I20260806 03:51:27.100958 12279 external_mini_cluster.cc:1428] Setting key b89fb50c86f356e830f2d0f8888a4357
Aug 06 03:51:27 dist-test-slave-6ntk krb5kdc[10991](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988287, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.253.253@KRBTEST.COM for kudu/127.11.253.254@KRBTEST.COM
I20260806 03:51:27.110095 11097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:27.100691 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52898 (local address 127.11.253.254:37231)
0806 03:51:27.100769 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0806 03:51:27.100771 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:51:27.100991 (+ 220us) server_negotiation.cc:407] Connection header received
0806 03:51:27.101531 (+ 540us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:51:27.101532 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:51:27.101548 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:51:27.101614 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:51:27.102373 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:27.102892 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:27.103882 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:27.104121 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:27.104203 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:27.106821 (+ 2618us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:51:27.106823 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:51:27.106823 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:51:27.106832 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:51:27.108523 (+ 1691us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:51:27.109026 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:51:27.109027 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:51:27.109027 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:51:27.109079 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:51:27.109382 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:51:27.109382 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:51:27.109383 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:51:27.109505 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:51:27.109556 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0806 03:51:27.109952 (+ 396us) server_negotiation.cc:299] Negotiation successful
0806 03:51:27.110026 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
WARNING: no policy specified for kudu/127.11.253.252@KRBTEST.COM; defaulting to no policy
I20260806 03:51:27.110376 11192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:27.100652 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:37231 (local address 127.0.0.1:52898)
0806 03:51:27.100846 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0806 03:51:27.100891 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 03:51:27.101419 (+ 528us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:51:27.101711 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:51:27.101715 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:51:27.102062 (+ 347us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:51:27.102266 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:51:27.102271 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:51:27.102998 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:51:27.102999 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:51:27.103784 (+ 785us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:51:27.103786 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:51:27.104212 (+ 426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:51:27.104213 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:51:27.104336 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:27.104914 (+ 578us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:51:27.104923 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:51:27.106700 (+ 1777us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:51:27.108610 (+ 1910us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:51:27.108617 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:51:27.108619 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:51:27.108946 (+ 327us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:51:27.109147 (+ 201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:51:27.109148 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:51:27.109148 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:51:27.109310 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:51:27.109578 (+ 268us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:51:27.109586 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:51:27.109765 (+ 179us) client_negotiation.cc:769] Sending connection context
0806 03:51:27.109911 (+ 146us) client_negotiation.cc:240] Negotiation successful
0806 03:51:27.110083 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":98,"threads_started":1}
I20260806 03:51:27.111140 11189 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.253.253" port: 36387 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.253.252@KRBTEST.COM" created.
Aug 06 03:51:27 dist-test-slave-6ntk krb5kdc[10991](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988287, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.253.253@KRBTEST.COM for kudu/127.11.253.253@KRBTEST.COM
I20260806 03:51:27.118716 11192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:27.111426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.253:36387 (local address 127.0.0.1:51308)
0806 03:51:27.111481 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0806 03:51:27.111493 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 03:51:27.111541 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:51:27.111802 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:51:27.111803 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:51:27.111941 (+ 138us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:51:27.111985 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:51:27.111986 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:51:27.112862 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:51:27.112863 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:51:27.113422 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:51:27.113424 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:51:27.113904 (+ 480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:51:27.113906 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:51:27.113971 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:27.114383 (+ 412us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:51:27.114387 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:51:27.115594 (+ 1207us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:51:27.117570 (+ 1976us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:51:27.117571 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:51:27.117571 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:51:27.117736 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:51:27.118009 (+ 273us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:51:27.118010 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:51:27.118010 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:51:27.118068 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:51:27.118389 (+ 321us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:51:27.118390 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:51:27.118430 (+ 40us) client_negotiation.cc:769] Sending connection context
0806 03:51:27.118497 (+ 67us) client_negotiation.cc:240] Negotiation successful
0806 03:51:27.118598 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260806 03:51:27.118782 11194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:27.111498 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51308 (local address 127.11.253.253:36387)
0806 03:51:27.111671 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0806 03:51:27.111675 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 03:51:27.111691 (+ 16us) server_negotiation.cc:407] Connection header received
0806 03:51:27.111707 (+ 16us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:51:27.111709 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:51:27.111719 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:51:27.111803 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:51:27.112062 (+ 259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:27.112770 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:27.113501 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:27.113810 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:27.113919 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:27.115694 (+ 1775us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:51:27.115702 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:51:27.115703 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:51:27.115709 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:51:27.117483 (+ 1774us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:51:27.117836 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:51:27.117839 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:51:27.117840 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:51:27.117943 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:51:27.118136 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:51:27.118137 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:51:27.118137 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:51:27.118291 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:51:27.118400 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0806 03:51:27.118533 (+ 133us) server_negotiation.cc:299] Negotiation successful
0806 03:51:27.118653 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":52,"server-negotiator.queue_time_us":140,"thread_start_us":89,"threads_started":1}
I20260806 03:51:27.119603 11189 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.253.252" port: 35087 } has no permanent_uuid. Determining permanent_uuid...
I20260806 03:51:27.119956 11192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:27.119834 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.252:35087 (local address 127.0.0.1:46606)
0806 03:51:27.119882 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0806 03:51:27.119926 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.253.252:35087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260806 03:51:27.120079 11126 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.11.253.252:35087: connect: Connection refused (error 111)
I20260806 03:51:27.120530 11192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:27.120436 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.252:35087 (local address 127.0.0.1:46610)
0806 03:51:27.120485 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0806 03:51:27.120509 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.253.252:35087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260806 03:51:27.120654 11189 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.253.252:35087: Network error: Client connection negotiation failed: client connection to 127.11.253.252:35087: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.253.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.252.keytab.
Entry for principal kudu/127.11.253.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.252.keytab.
I20260806 03:51:27.128748 12279 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskToVWRe/build/release/bin/kudu
/tmp/dist-test-taskToVWRe/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785988004637995-12279-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785988004637995-12279-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785988004637995-12279-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785988004637995-12279-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.253.252:35087
--webserver_interface=127.11.253.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.252.keytab
--principal=kudu/127.11.253.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.253.212:40457
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.253.254:37231,127.11.253.253:36387,127.11.253.252:35087
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785988004637995-12279-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785988004637995-12279-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785988004637995-12279-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 03:51:27.155625 11093 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.253.253" port: 36387 } attempt: 3
Aug 06 03:51:27 dist-test-slave-6ntk krb5kdc[10991](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988286, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.253.254@KRBTEST.COM for kudu/127.11.253.253@KRBTEST.COM
I20260806 03:51:27.162866 11194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:27.156146 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51312 (local address 127.11.253.253:36387)
0806 03:51:27.156228 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0806 03:51:27.156231 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 03:51:27.156244 (+ 13us) server_negotiation.cc:407] Connection header received
0806 03:51:27.156277 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:51:27.156278 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:51:27.156292 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:51:27.156351 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:51:27.156684 (+ 333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:27.157196 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:27.157975 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:27.158190 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:27.158273 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:27.160176 (+ 1903us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:51:27.160177 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:51:27.160177 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:51:27.160184 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:51:27.161630 (+ 1446us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:51:27.162067 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:51:27.162068 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:51:27.162069 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:51:27.162129 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:51:27.162336 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:51:27.162337 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:51:27.162337 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:51:27.162455 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:51:27.162528 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0806 03:51:27.162684 (+ 156us) server_negotiation.cc:299] Negotiation successful
0806 03:51:27.162774 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260806 03:51:27.162894 11096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:27.156053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.253:36387 (local address 127.0.0.1:51312)
0806 03:51:27.156122 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0806 03:51:27.156138 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 03:51:27.156218 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:51:27.156366 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:51:27.156367 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:51:27.156537 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:51:27.156599 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:51:27.156601 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:51:27.157288 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:51:27.157289 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:51:27.157830 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:51:27.157832 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:51:27.158276 (+ 444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:51:27.158277 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:51:27.158362 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:27.158850 (+ 488us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:51:27.158856 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:51:27.160085 (+ 1229us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:51:27.161734 (+ 1649us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:51:27.161735 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:51:27.161736 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:51:27.161985 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:51:27.162207 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:51:27.162208 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:51:27.162208 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:51:27.162273 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:51:27.162531 (+ 258us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:51:27.162532 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:51:27.162578 (+ 46us) client_negotiation.cc:769] Sending connection context
0806 03:51:27.162659 (+ 81us) client_negotiation.cc:240] Negotiation successful
0806 03:51:27.162765 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42}
I20260806 03:51:27.163467 11093 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.253.252" port: 35087 } has no permanent_uuid. Determining permanent_uuid...
I20260806 03:51:27.163815 11096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:27.163714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.252:35087 (local address 127.0.0.1:46612)
0806 03:51:27.163761 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0806 03:51:27.163790 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.253.252:35087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260806 03:51:27.164186 11096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:27.164127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.252:35087 (local address 127.0.0.1:46624)
0806 03:51:27.164150 (+ 23us) negotiation.cc:107] Waiting for socket to connect
0806 03:51:27.164172 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.253.252:35087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":15}
W20260806 03:51:27.164278 11093 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.253.252:35087: Network error: Client connection negotiation failed: client connection to 127.11.253.252:35087: connect: Connection refused (error 111)
I20260806 03:51:27.169919 11189 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.253.252" port: 35087 } attempt: 1
I20260806 03:51:27.170337 11192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:27.170209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.252:35087 (local address 127.0.0.1:46632)
0806 03:51:27.170270 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0806 03:51:27.170307 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.253.252:35087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260806 03:51:27.170665 11192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:27.170576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.252:35087 (local address 127.0.0.1:46640)
0806 03:51:27.170619 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0806 03:51:27.170644 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.253.252:35087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260806 03:51:27.170764 11189 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.253.252:35087: Network error: Client connection negotiation failed: client connection to 127.11.253.252:35087: connect: Connection refused (error 111)
I20260806 03:51:27.195418 11093 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.253.252" port: 35087 } attempt: 1
I20260806 03:51:27.195917 11096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:27.195755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.252:35087 (local address 127.0.0.1:46644)
0806 03:51:27.195843 (+ 88us) negotiation.cc:107] Waiting for socket to connect
0806 03:51:27.195884 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.253.252:35087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260806 03:51:27.196326 11096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:27.196182 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.252:35087 (local address 127.0.0.1:46648)
0806 03:51:27.196231 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0806 03:51:27.196258 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.253.252:35087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260806 03:51:27.196424 11093 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.253.252:35087: Network error: Client connection negotiation failed: client connection to 127.11.253.252:35087: connect: Connection refused (error 111)
W20260806 03:51:27.215329 11198 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 03:51:27.215553 11198 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:51:27.215571 11198 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:51:27.215619 11198 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:51:27.216908 11198 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 03:51:27.216952 11198 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:51:27.216967 11198 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:51:27.216980 11198 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 03:51:27.216992 11198 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 03:51:27.218588 11198 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.253.212:40457
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785988004637995-12279-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785988004637995-12279-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.11.253.254:37231,127.11.253.253:36387,127.11.253.252:35087
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.252.keytab
--principal=kudu/127.11.253.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.253.252:35087
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785988004637995-12279-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.11.253.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11198
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785988004637995-12279-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:51:27.218845 11198 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:51:27.219103 11198 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:51:27.224252 11209 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:51:27.224202 11205 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:51:27.224406 11204 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:51:27.224638 11198 server_base.cc:1034] running on GCE node
I20260806 03:51:27.224941 11198 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:51:27.225229 11198 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:51:27.226387 11198 hybrid_clock.cc:648] HybridClock initialized: now 1785988287226370 us; error 32 us; skew 500 ppm
Aug 06 03:51:27 dist-test-slave-6ntk krb5kdc[10991](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988287, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.253.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:51:27.230048 11198 init.cc:377] Logged in from keytab as kudu/127.11.253.252@KRBTEST.COM (short username kudu)
I20260806 03:51:27.230545 11198 webserver.cc:460] Webserver started at http://127.11.253.252:44567/ using document root <none> and password file <none>
I20260806 03:51:27.230830 11198 fs_manager.cc:359] Metadata directory not provided
I20260806 03:51:27.230883 11198 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:51:27.231060 11198 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:51:27.232241 11198 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785988004637995-12279-0/minicluster-data/master-2/data/instance:
uuid: "a3b7c28c24d34118a81fbe40b2ff06fb"
format_stamp: "Formatted at 2026-08-06 03:51:27 on dist-test-slave-6ntk"
server_key: "468d65de5564198df6493646e4b56616"
server_key_iv: "8d554a535273a5f2cfb9d878e01320cb"
server_key_version: "encryptionkey@0"
I20260806 03:51:27.232657 11198 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785988004637995-12279-0/minicluster-data/master-2/wal/instance:
uuid: "a3b7c28c24d34118a81fbe40b2ff06fb"
format_stamp: "Formatted at 2026-08-06 03:51:27 on dist-test-slave-6ntk"
server_key: "468d65de5564198df6493646e4b56616"
server_key_iv: "8d554a535273a5f2cfb9d878e01320cb"
server_key_version: "encryptionkey@0"
I20260806 03:51:27.234220 11198 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.001s
I20260806 03:51:27.235154 11216 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:51:27.235347 11198 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 03:51:27.235445 11198 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785988004637995-12279-0/minicluster-data/master-2/data,/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785988004637995-12279-0/minicluster-data/master-2/wal
uuid: "a3b7c28c24d34118a81fbe40b2ff06fb"
format_stamp: "Formatted at 2026-08-06 03:51:27 on dist-test-slave-6ntk"
server_key: "468d65de5564198df6493646e4b56616"
server_key_iv: "8d554a535273a5f2cfb9d878e01320cb"
server_key_version: "encryptionkey@0"
I20260806 03:51:27.235529 11198 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785988004637995-12279-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785988004637995-12279-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785988004637995-12279-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:51:27.238909 11189 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.253.252" port: 35087 } attempt: 2
I20260806 03:51:27.239344 11192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:27.239205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.252:35087 (local address 127.0.0.1:46662)
0806 03:51:27.239264 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0806 03:51:27.239301 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.253.252:35087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260806 03:51:27.239725 11192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:27.239629 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.252:35087 (local address 127.0.0.1:46664)
0806 03:51:27.239675 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0806 03:51:27.239700 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.253.252:35087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260806 03:51:27.239830 11189 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.253.252:35087: Network error: Client connection negotiation failed: client connection to 127.11.253.252:35087: connect: Connection refused (error 111)
I20260806 03:51:27.244058 11198 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:51:27.246043 11198 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:51:27.246222 11198 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:51:27.250505 11198 rpc_server.cc:307] RPC server started. Bound to: 127.11.253.252:35087
I20260806 03:51:27.250535 11292 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.253.252:35087 every 8 connection(s)
I20260806 03:51:27.250932 11198 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785988004637995-12279-0/minicluster-data/master-2/data/info.pb
I20260806 03:51:27.251814 11294 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:51:27.253428 11294 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.253.254" port: 37231 } has no permanent_uuid. Determining permanent_uuid...
I20260806 03:51:27.254333 12279 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskToVWRe/build/release/bin/kudu as pid 11198
I20260806 03:51:27.254434 12279 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785988004637995-12279-0/minicluster-data/master-2/wal/instance
I20260806 03:51:27.254631 12279 external_mini_cluster.cc:1428] Setting key 6ca74ff47f4e33a7dc631c6cce9f4c3c
Aug 06 03:51:27 dist-test-slave-6ntk krb5kdc[10991](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988287, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.253.252@KRBTEST.COM for kudu/127.11.253.254@KRBTEST.COM
Aug 06 03:51:27 dist-test-slave-6ntk krb5kdc[10991](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988286, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.253.252@KRBTEST.COM
I20260806 03:51:27.264859 11097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:27.254187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52910 (local address 127.11.253.254:37231)
0806 03:51:27.254253 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0806 03:51:27.254255 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:51:27.254540 (+ 285us) server_negotiation.cc:407] Connection header received
0806 03:51:27.255395 (+ 855us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:51:27.255397 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:51:27.255422 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:51:27.255494 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:51:27.256367 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:27.257132 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:27.258455 (+ 1323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:27.258697 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:27.258769 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:27.261393 (+ 2624us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:51:27.261395 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:51:27.261395 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:51:27.261404 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:51:27.263153 (+ 1749us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:51:27.263747 (+ 594us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:51:27.263748 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:51:27.263748 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:51:27.263800 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:51:27.264098 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:51:27.264100 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:51:27.264100 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:51:27.264239 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:51:27.264326 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0806 03:51:27.264651 (+ 325us) server_negotiation.cc:299] Negotiation successful
0806 03:51:27.264744 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260806 03:51:27.265053 11296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:27.254120 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:37231 (local address 127.0.0.1:52910)
0806 03:51:27.254377 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0806 03:51:27.254449 (+ 72us) client_negotiation.cc:174] Beginning negotiation
0806 03:51:27.255253 (+ 804us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:51:27.255556 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:51:27.255559 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:51:27.255972 (+ 413us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:51:27.256241 (+ 269us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:51:27.256248 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:51:27.257221 (+ 973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:51:27.257222 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:51:27.258365 (+ 1143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:51:27.258368 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:51:27.258788 (+ 420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:51:27.258790 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:51:27.258948 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:27.259529 (+ 581us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:51:27.259542 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:51:27.261269 (+ 1727us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:51:27.263239 (+ 1970us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:51:27.263245 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:51:27.263248 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:51:27.263669 (+ 421us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:51:27.263870 (+ 201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:51:27.263871 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:51:27.263871 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:51:27.264016 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:51:27.264321 (+ 305us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:51:27.264329 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:51:27.264484 (+ 155us) client_negotiation.cc:769] Sending connection context
0806 03:51:27.264612 (+ 128us) client_negotiation.cc:240] Negotiation successful
0806 03:51:27.264779 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":197,"mutex_wait_us":27,"thread_start_us":80,"threads_started":1}
I20260806 03:51:27.265061 11300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:27.255098 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46670 (local address 127.11.253.252:35087)
0806 03:51:27.255257 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0806 03:51:27.255261 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 03:51:27.255351 (+ 90us) server_negotiation.cc:407] Connection header received
0806 03:51:27.255458 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:51:27.255460 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:51:27.255481 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:51:27.255567 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:51:27.256402 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:27.257496 (+ 1094us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:27.258614 (+ 1118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:27.259075 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:27.259130 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:27.261863 (+ 2733us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:51:27.261929 (+ 66us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:51:27.261930 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:51:27.261941 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:51:27.263814 (+ 1873us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:51:27.264170 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:51:27.264173 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:51:27.264174 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:51:27.264243 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:51:27.264462 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:51:27.264463 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:51:27.264463 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:51:27.264611 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:51:27.264789 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0806 03:51:27.264835 (+ 46us) server_negotiation.cc:299] Negotiation successful
0806 03:51:27.264929 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":35,"server-negotiator.queue_time_us":130,"thread_start_us":80,"threads_started":1}
I20260806 03:51:27.266057 11294 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.253.253" port: 36387 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 03:51:27 dist-test-slave-6ntk krb5kdc[10991](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988287, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.253.252@KRBTEST.COM for kudu/127.11.253.253@KRBTEST.COM
Aug 06 03:51:27 dist-test-slave-6ntk krb5kdc[10991](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988286, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.253.254@KRBTEST.COM
I20260806 03:51:27.271545 11093 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.253.252" port: 35087 } attempt: 2
I20260806 03:51:27.273340 11296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:27.266332 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.253:36387 (local address 127.0.0.1:51318)
0806 03:51:27.266408 (+ 76us) negotiation.cc:107] Waiting for socket to connect
0806 03:51:27.266421 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 03:51:27.266457 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:51:27.266577 (+ 120us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:51:27.266578 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:51:27.266750 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:51:27.266819 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:51:27.266821 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:51:27.267436 (+ 615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:51:27.267437 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:51:27.267905 (+ 468us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:51:27.267907 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:51:27.268299 (+ 392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:51:27.268300 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:51:27.268374 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:27.268787 (+ 413us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:51:27.268792 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:51:27.270122 (+ 1330us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:51:27.272075 (+ 1953us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:51:27.272076 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:51:27.272076 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:51:27.272303 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:51:27.272637 (+ 334us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:51:27.272639 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:51:27.272639 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:51:27.272713 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:51:27.273003 (+ 290us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:51:27.273004 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:51:27.273044 (+ 40us) client_negotiation.cc:769] Sending connection context
0806 03:51:27.273132 (+ 88us) client_negotiation.cc:240] Negotiation successful
0806 03:51:27.273235 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":57}
I20260806 03:51:27.273386 11194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:27.266393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51318 (local address 127.11.253.253:36387)
0806 03:51:27.266445 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0806 03:51:27.266447 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:51:27.266456 (+ 9us) server_negotiation.cc:407] Connection header received
0806 03:51:27.266516 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:51:27.266516 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:51:27.266525 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:51:27.266568 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:51:27.266896 (+ 328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:27.267350 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:27.267994 (+ 644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:27.268214 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:27.268289 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:27.270224 (+ 1935us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:51:27.270226 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:51:27.270226 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:51:27.270235 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:51:27.271991 (+ 1756us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:51:27.272510 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:51:27.272511 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:51:27.272512 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:51:27.272577 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:51:27.272786 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:51:27.272787 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:51:27.272787 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:51:27.272937 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:51:27.273003 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0806 03:51:27.273167 (+ 164us) server_negotiation.cc:299] Negotiation successful
0806 03:51:27.273249 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260806 03:51:27.274374 11294 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.253.252" port: 35087 } has no permanent_uuid. Determining permanent_uuid...
I20260806 03:51:27.274456 11097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:27.266171 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52916 (local address 127.11.253.254:37231)
0806 03:51:27.266211 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0806 03:51:27.266213 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:51:27.266221 (+ 8us) server_negotiation.cc:407] Connection header received
0806 03:51:27.266287 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:51:27.266288 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:51:27.266298 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:51:27.266361 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:51:27.267042 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:27.267547 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:27.268280 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:27.268509 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:27.268592 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:27.270701 (+ 2109us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:51:27.270702 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:51:27.270702 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:51:27.270709 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:51:27.272708 (+ 1999us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:51:27.273333 (+ 625us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:51:27.273334 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:51:27.273334 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:51:27.273386 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:51:27.273587 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:51:27.273589 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:51:27.273589 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:51:27.273728 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:51:27.273970 (+ 242us) server_negotiation.cc:1035] Waiting for connection context
0806 03:51:27.274226 (+ 256us) server_negotiation.cc:299] Negotiation successful
0806 03:51:27.274333 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
Aug 06 03:51:27 dist-test-slave-6ntk krb5kdc[10991](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988286, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.253.254@KRBTEST.COM for kudu/127.11.253.252@KRBTEST.COM
Aug 06 03:51:27 dist-test-slave-6ntk krb5kdc[10991](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988287, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.253.252@KRBTEST.COM for kudu/127.11.253.252@KRBTEST.COM
Aug 06 03:51:27 dist-test-slave-6ntk krb5kdc[10991](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988286, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.253.253@KRBTEST.COM
I20260806 03:51:27.280848 11096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:27.271816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.252:35087 (local address 127.0.0.1:46678)
0806 03:51:27.271875 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0806 03:51:27.271887 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 03:51:27.271932 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:51:27.272114 (+ 182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:51:27.272116 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:51:27.272370 (+ 254us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:51:27.272404 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:51:27.272406 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:51:27.273281 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:51:27.273282 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:51:27.274674 (+ 1392us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:51:27.274677 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:51:27.275163 (+ 486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:51:27.275164 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:51:27.275232 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:27.275672 (+ 440us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:51:27.275677 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:51:27.277112 (+ 1435us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:51:27.279420 (+ 2308us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:51:27.279421 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:51:27.279422 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:51:27.279739 (+ 317us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:51:27.279994 (+ 255us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:51:27.279995 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:51:27.279995 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:51:27.280066 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:51:27.280443 (+ 377us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:51:27.280444 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:51:27.280497 (+ 53us) client_negotiation.cc:769] Sending connection context
0806 03:51:27.280579 (+ 82us) client_negotiation.cc:240] Negotiation successful
0806 03:51:27.280723 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40}
I20260806 03:51:27.280931 11300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:27.271871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46678 (local address 127.11.253.252:35087)
0806 03:51:27.271912 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0806 03:51:27.271914 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:51:27.271923 (+ 9us) server_negotiation.cc:407] Connection header received
0806 03:51:27.272003 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:51:27.272004 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:51:27.272014 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:51:27.272099 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:51:27.272481 (+ 382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:27.273184 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:27.274764 (+ 1580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:27.275074 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:27.275139 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:27.277203 (+ 2064us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:51:27.277205 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:51:27.277205 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:51:27.277213 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:51:27.279292 (+ 2079us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:51:27.279839 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:51:27.279840 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:51:27.279840 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:51:27.279902 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:51:27.280173 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:51:27.280174 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:51:27.280175 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:51:27.280337 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:51:27.280443 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0806 03:51:27.280674 (+ 231us) server_negotiation.cc:299] Negotiation successful
0806 03:51:27.280788 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 03:51:27.283241 11093 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3b37dc1a270849ebbed01fe4798c3c01: Bootstrap starting.
I20260806 03:51:27.283365 11296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:27.274596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.252:35087 (local address 127.0.0.1:46682)
0806 03:51:27.274652 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0806 03:51:27.274664 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 03:51:27.274710 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:51:27.274990 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:51:27.274991 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:51:27.275181 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:51:27.275212 (+ 31us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:51:27.275213 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:51:27.276167 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:51:27.276168 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:51:27.276928 (+ 760us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:51:27.276931 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:51:27.277384 (+ 453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:51:27.277385 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:51:27.277445 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:27.277932 (+ 487us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:51:27.277937 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:51:27.279195 (+ 1258us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:51:27.282120 (+ 2925us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:51:27.282121 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:51:27.282121 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:51:27.282323 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:51:27.282572 (+ 249us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:51:27.282573 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:51:27.282573 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:51:27.282644 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:51:27.282960 (+ 316us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:51:27.282961 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:51:27.283004 (+ 43us) client_negotiation.cc:769] Sending connection context
0806 03:51:27.283100 (+ 96us) client_negotiation.cc:240] Negotiation successful
0806 03:51:27.283210 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260806 03:51:27.283713 11301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:27.274660 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46682 (local address 127.11.253.252:35087)
0806 03:51:27.274848 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0806 03:51:27.274850 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:51:27.274857 (+ 7us) server_negotiation.cc:407] Connection header received
0806 03:51:27.274876 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:51:27.274877 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:51:27.274889 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:51:27.274982 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:51:27.275299 (+ 317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:27.276068 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:27.277017 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:27.277307 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:27.277387 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:27.279378 (+ 1991us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:51:27.279379 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:51:27.279380 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:51:27.279386 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:51:27.282014 (+ 2628us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:51:27.282393 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:51:27.282394 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:51:27.282394 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:51:27.282463 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:51:27.282717 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:51:27.282718 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:51:27.282718 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:51:27.282874 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:51:27.282958 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0806 03:51:27.283127 (+ 169us) server_negotiation.cc:299] Negotiation successful
0806 03:51:27.283612 (+ 485us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":413,"server-negotiator.queue_time_us":166,"thread_start_us":123,"threads_started":1}
I20260806 03:51:27.284153 11093 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3b37dc1a270849ebbed01fe4798c3c01: Neither blocks nor log segments found. Creating new log.
I20260806 03:51:27.284550 11093 log.cc:824] T 00000000000000000000000000000000 P 3b37dc1a270849ebbed01fe4798c3c01: Log is configured to *not* fsync() on all Append() calls
I20260806 03:51:27.284847 11194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:27.275468 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51320 (local address 127.11.253.253:36387)
0806 03:51:27.275507 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0806 03:51:27.275509 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:51:27.275518 (+ 9us) server_negotiation.cc:407] Connection header received
0806 03:51:27.275591 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:51:27.275591 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:51:27.275600 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:51:27.275644 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:51:27.276165 (+ 521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:27.276656 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:27.277567 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:27.278000 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:27.278090 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:27.281150 (+ 3060us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:51:27.281151 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:51:27.281151 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:51:27.281160 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:51:27.283152 (+ 1992us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:51:27.283729 (+ 577us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:51:27.283730 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:51:27.283731 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:51:27.283800 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:51:27.284030 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:51:27.284031 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:51:27.284031 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:51:27.284170 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:51:27.284354 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0806 03:51:27.284628 (+ 274us) server_negotiation.cc:299] Negotiation successful
0806 03:51:27.284731 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260806 03:51:27.285406 11294 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a3b7c28c24d34118a81fbe40b2ff06fb: Bootstrap starting.
I20260806 03:51:27.285588 11093 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3b37dc1a270849ebbed01fe4798c3c01: No bootstrap required, opened a new log
I20260806 03:51:27.286273 11294 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a3b7c28c24d34118a81fbe40b2ff06fb: Neither blocks nor log segments found. Creating new log.
I20260806 03:51:27.286648 11294 log.cc:824] T 00000000000000000000000000000000 P a3b7c28c24d34118a81fbe40b2ff06fb: Log is configured to *not* fsync() on all Append() calls
I20260806 03:51:27.286957 12279 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 03:51:27.287539 11294 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a3b7c28c24d34118a81fbe40b2ff06fb: No bootstrap required, opened a new log
I20260806 03:51:27.287766 11093 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3b37dc1a270849ebbed01fe4798c3c01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b37dc1a270849ebbed01fe4798c3c01" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 37231 } } peers { permanent_uuid: "b201733feb5f4a5c87ebf08707f39461" member_type: VOTER last_known_addr { host: "127.11.253.253" port: 36387 } } peers { permanent_uuid: "a3b7c28c24d34118a81fbe40b2ff06fb" member_type: VOTER last_known_addr { host: "127.11.253.252" port: 35087 } }
I20260806 03:51:27.287916 11093 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3b37dc1a270849ebbed01fe4798c3c01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b37dc1a270849ebbed01fe4798c3c01, State: Initialized, Role: FOLLOWER
I20260806 03:51:27.288025 11093 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3b37dc1a270849ebbed01fe4798c3c01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b37dc1a270849ebbed01fe4798c3c01" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 37231 } } peers { permanent_uuid: "b201733feb5f4a5c87ebf08707f39461" member_type: VOTER last_known_addr { host: "127.11.253.253" port: 36387 } } peers { permanent_uuid: "a3b7c28c24d34118a81fbe40b2ff06fb" member_type: VOTER last_known_addr { host: "127.11.253.252" port: 35087 } }
I20260806 03:51:27.288116 11062 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "3b37dc1a270849ebbed01fe4798c3c01"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52916
I20260806 03:51:27.288383 11093 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3b37dc1a270849ebbed01fe4798c3c01 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 03:51:27.288481 11062 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3b37dc1a270849ebbed01fe4798c3c01 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260806 03:51:27.288537 11062 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3b37dc1a270849ebbed01fe4798c3c01 [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:51:27.289505 11062 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3b37dc1a270849ebbed01fe4798c3c01 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b37dc1a270849ebbed01fe4798c3c01" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 37231 } } peers { permanent_uuid: "b201733feb5f4a5c87ebf08707f39461" member_type: VOTER last_known_addr { host: "127.11.253.253" port: 36387 } } peers { permanent_uuid: "a3b7c28c24d34118a81fbe40b2ff06fb" member_type: VOTER last_known_addr { host: "127.11.253.252" port: 35087 } }
I20260806 03:51:27.289748 11294 raft_consensus.cc:348] T 00000000000000000000000000000000 P a3b7c28c24d34118a81fbe40b2ff06fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b37dc1a270849ebbed01fe4798c3c01" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 37231 } } peers { permanent_uuid: "b201733feb5f4a5c87ebf08707f39461" member_type: VOTER last_known_addr { host: "127.11.253.253" port: 36387 } } peers { permanent_uuid: "a3b7c28c24d34118a81fbe40b2ff06fb" member_type: VOTER last_known_addr { host: "127.11.253.252" port: 35087 } }
I20260806 03:51:27.289934 11294 raft_consensus.cc:729] T 00000000000000000000000000000000 P a3b7c28c24d34118a81fbe40b2ff06fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3b7c28c24d34118a81fbe40b2ff06fb, State: Initialized, Role: FOLLOWER
I20260806 03:51:27.290019 11294 consensus_queue.cc:260] T 00000000000000000000000000000000 P a3b7c28c24d34118a81fbe40b2ff06fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b37dc1a270849ebbed01fe4798c3c01" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 37231 } } peers { permanent_uuid: "b201733feb5f4a5c87ebf08707f39461" member_type: VOTER last_known_addr { host: "127.11.253.253" port: 36387 } } peers { permanent_uuid: "a3b7c28c24d34118a81fbe40b2ff06fb" member_type: VOTER last_known_addr { host: "127.11.253.252" port: 35087 } }
I20260806 03:51:27.290323 11294 sys_catalog.cc:564] T 00000000000000000000000000000000 P a3b7c28c24d34118a81fbe40b2ff06fb [sys.catalog]: configured and running, proceeding with master startup.
W20260806 03:51:27.290357 11321 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3b37dc1a270849ebbed01fe4798c3c01: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 03:51:27.290403 11321 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 03:51:27.290357 11256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3b37dc1a270849ebbed01fe4798c3c01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "a3b7c28c24d34118a81fbe40b2ff06fb"
I20260806 03:51:27.289779 11062 leader_election.cc:288] T 00000000000000000000000000000000 P 3b37dc1a270849ebbed01fe4798c3c01 [CANDIDATE]: Term 1 election: Requested vote from peers b201733feb5f4a5c87ebf08707f39461 (127.11.253.253:36387), a3b7c28c24d34118a81fbe40b2ff06fb (127.11.253.252:35087)
I20260806 03:51:27.290495 11256 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a3b7c28c24d34118a81fbe40b2ff06fb [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:51:27.290916 11304 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3b37dc1a270849ebbed01fe4798c3c01 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b37dc1a270849ebbed01fe4798c3c01" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 37231 } } peers { permanent_uuid: "b201733feb5f4a5c87ebf08707f39461" member_type: VOTER last_known_addr { host: "127.11.253.253" port: 36387 } } peers { permanent_uuid: "a3b7c28c24d34118a81fbe40b2ff06fb" member_type: VOTER last_known_addr { host: "127.11.253.252" port: 35087 } } }
I20260806 03:51:27.291002 11304 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3b37dc1a270849ebbed01fe4798c3c01 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 03:51:27.291318 11256 raft_consensus.cc:2454] T 00000000000000000000000000000000 P a3b7c28c24d34118a81fbe40b2ff06fb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b37dc1a270849ebbed01fe4798c3c01 in term 1.
I20260806 03:51:27.291553 11028 leader_election.cc:302] T 00000000000000000000000000000000 P 3b37dc1a270849ebbed01fe4798c3c01 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b37dc1a270849ebbed01fe4798c3c01, a3b7c28c24d34118a81fbe40b2ff06fb; no voters:
I20260806 03:51:27.291677 11304 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3b37dc1a270849ebbed01fe4798c3c01 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 03:51:27.291819 11304 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3b37dc1a270849ebbed01fe4798c3c01 [term 1 LEADER]: Becoming Leader. State: Replica: 3b37dc1a270849ebbed01fe4798c3c01, State: Running, Role: LEADER
I20260806 03:51:27.291919 11304 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3b37dc1a270849ebbed01fe4798c3c01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b37dc1a270849ebbed01fe4798c3c01" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 37231 } } peers { permanent_uuid: "b201733feb5f4a5c87ebf08707f39461" member_type: VOTER last_known_addr { host: "127.11.253.253" port: 36387 } } peers { permanent_uuid: "a3b7c28c24d34118a81fbe40b2ff06fb" member_type: VOTER last_known_addr { host: "127.11.253.252" port: 35087 } }
I20260806 03:51:27.292389 11304 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3b37dc1a270849ebbed01fe4798c3c01 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3b37dc1a270849ebbed01fe4798c3c01. Latest consensus state: current_term: 1 leader_uuid: "3b37dc1a270849ebbed01fe4798c3c01" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b37dc1a270849ebbed01fe4798c3c01" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 37231 } } peers { permanent_uuid: "b201733feb5f4a5c87ebf08707f39461" member_type: VOTER last_known_addr { host: "127.11.253.253" port: 36387 } } peers { permanent_uuid: "a3b7c28c24d34118a81fbe40b2ff06fb" member_type: VOTER last_known_addr { host: "127.11.253.252" port: 35087 } } }
I20260806 03:51:27.292533 11304 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3b37dc1a270849ebbed01fe4798c3c01 [sys.catalog]: This master's current role is: LEADER
I20260806 03:51:27.291975 11009 master_runner.cc:186] Error getting master registration for 127.11.253.253:36387: OK, instance_id { permanent_uuid: "b201733feb5f4a5c87ebf08707f39461" instance_seqno: 1785988287095167 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 03:51:27.294950 11009 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 03:51:27.297983 11323 sys_catalog.cc:455] T 00000000000000000000000000000000 P a3b7c28c24d34118a81fbe40b2ff06fb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b37dc1a270849ebbed01fe4798c3c01" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 37231 } } peers { permanent_uuid: "b201733feb5f4a5c87ebf08707f39461" member_type: VOTER last_known_addr { host: "127.11.253.253" port: 36387 } } peers { permanent_uuid: "a3b7c28c24d34118a81fbe40b2ff06fb" member_type: VOTER last_known_addr { host: "127.11.253.252" port: 35087 } } }
I20260806 03:51:27.298091 11323 sys_catalog.cc:458] T 00000000000000000000000000000000 P a3b7c28c24d34118a81fbe40b2ff06fb [sys.catalog]: This master's current role is: FOLLOWER
I20260806 03:51:27.295428 11198 master_runner.cc:186] Error getting master registration for 127.11.253.253:36387: OK, instance_id { permanent_uuid: "b201733feb5f4a5c87ebf08707f39461" instance_seqno: 1785988287095167 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 03:51:27.298569 11198 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260806 03:51:27.298774 11337 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a3b7c28c24d34118a81fbe40b2ff06fb: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 03:51:27.298830 11337 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 03:51:27.299010 11346 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 03:51:27.299192 11346 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 03:51:27.303043 11097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:27.292676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52932 (local address 127.11.253.254:37231)
0806 03:51:27.292759 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0806 03:51:27.292761 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:51:27.293997 (+ 1236us) server_negotiation.cc:407] Connection header received
0806 03:51:27.294093 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:51:27.294094 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:51:27.294104 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:51:27.294133 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:51:27.294929 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:27.295620 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:27.296648 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:27.296945 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:27.296990 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:27.298736 (+ 1746us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:51:27.298737 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:51:27.298738 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:51:27.298745 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:51:27.300765 (+ 2020us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:51:27.301490 (+ 725us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:51:27.301491 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:51:27.301491 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:51:27.301556 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:51:27.301990 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:51:27.301991 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:51:27.301991 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:51:27.302136 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:51:27.302220 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0806 03:51:27.302840 (+ 620us) server_negotiation.cc:299] Negotiation successful
0806 03:51:27.302930 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260806 03:51:27.307855 11194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:27.293686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51326 (local address 127.11.253.253:36387)
0806 03:51:27.293724 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0806 03:51:27.293726 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:51:27.299937 (+ 6211us) server_negotiation.cc:407] Connection header received
0806 03:51:27.299999 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:51:27.300000 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:51:27.300011 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:51:27.300059 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:51:27.300659 (+ 600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:27.301367 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:27.302423 (+ 1056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:27.302722 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:27.302766 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:27.304557 (+ 1791us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:51:27.304558 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:51:27.304558 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:51:27.304565 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:51:27.306496 (+ 1931us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:51:27.306944 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:51:27.306945 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:51:27.306945 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:51:27.307003 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:51:27.307225 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:51:27.307226 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:51:27.307226 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:51:27.307362 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:51:27.307455 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0806 03:51:27.307676 (+ 221us) server_negotiation.cc:299] Negotiation successful
0806 03:51:27.307761 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260806 03:51:27.309255 11301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:27.292790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46696 (local address 127.11.253.252:35087)
0806 03:51:27.292830 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0806 03:51:27.292833 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 03:51:27.293537 (+ 704us) server_negotiation.cc:407] Connection header received
0806 03:51:27.293704 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:51:27.293718 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:51:27.293730 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:51:27.293763 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:51:27.295273 (+ 1510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:27.296101 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:27.297856 (+ 1755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:27.298210 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:27.298259 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:27.305268 (+ 7009us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:51:27.305270 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:51:27.305270 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:51:27.305278 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:51:27.307262 (+ 1984us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:51:27.307949 (+ 687us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:51:27.307950 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:51:27.307951 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:51:27.308013 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:51:27.308413 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:51:27.308415 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:51:27.308415 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:51:27.308562 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:51:27.308646 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0806 03:51:27.309043 (+ 397us) server_negotiation.cc:299] Negotiation successful
0806 03:51:27.309145 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260806 03:51:27.309633 11256 raft_consensus.cc:1261] T 00000000000000000000000000000000 P a3b7c28c24d34118a81fbe40b2ff06fb [term 1 FOLLOWER]: Refusing update from remote peer 3b37dc1a270849ebbed01fe4798c3c01: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 03:51:27.310000 11338 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 3b37dc1a270849ebbed01fe4798c3c01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3b7c28c24d34118a81fbe40b2ff06fb" member_type: VOTER last_known_addr { host: "127.11.253.252" port: 35087 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 03:51:27.311416 11323 sys_catalog.cc:455] T 00000000000000000000000000000000 P a3b7c28c24d34118a81fbe40b2ff06fb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3b37dc1a270849ebbed01fe4798c3c01. Latest consensus state: current_term: 1 leader_uuid: "3b37dc1a270849ebbed01fe4798c3c01" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b37dc1a270849ebbed01fe4798c3c01" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 37231 } } peers { permanent_uuid: "b201733feb5f4a5c87ebf08707f39461" member_type: VOTER last_known_addr { host: "127.11.253.253" port: 36387 } } peers { permanent_uuid: "a3b7c28c24d34118a81fbe40b2ff06fb" member_type: VOTER last_known_addr { host: "127.11.253.252" port: 35087 } } }
I20260806 03:51:27.311520 11323 sys_catalog.cc:458] T 00000000000000000000000000000000 P a3b7c28c24d34118a81fbe40b2ff06fb [sys.catalog]: This master's current role is: FOLLOWER
I20260806 03:51:27.317112 11304 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3b37dc1a270849ebbed01fe4798c3c01 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "3b37dc1a270849ebbed01fe4798c3c01" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b37dc1a270849ebbed01fe4798c3c01" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 37231 } } peers { permanent_uuid: "b201733feb5f4a5c87ebf08707f39461" member_type: VOTER last_known_addr { host: "127.11.253.253" port: 36387 } } peers { permanent_uuid: "a3b7c28c24d34118a81fbe40b2ff06fb" member_type: VOTER last_known_addr { host: "127.11.253.252" port: 35087 } } }
I20260806 03:51:27.317215 11304 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3b37dc1a270849ebbed01fe4798c3c01 [sys.catalog]: This master's current role is: LEADER
I20260806 03:51:27.317260 11323 sys_catalog.cc:455] T 00000000000000000000000000000000 P a3b7c28c24d34118a81fbe40b2ff06fb [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "3b37dc1a270849ebbed01fe4798c3c01" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b37dc1a270849ebbed01fe4798c3c01" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 37231 } } peers { permanent_uuid: "b201733feb5f4a5c87ebf08707f39461" member_type: VOTER last_known_addr { host: "127.11.253.253" port: 36387 } } peers { permanent_uuid: "a3b7c28c24d34118a81fbe40b2ff06fb" member_type: VOTER last_known_addr { host: "127.11.253.252" port: 35087 } } }
I20260806 03:51:27.317358 11323 sys_catalog.cc:458] T 00000000000000000000000000000000 P a3b7c28c24d34118a81fbe40b2ff06fb [sys.catalog]: This master's current role is: FOLLOWER
I20260806 03:51:27.318095 11358 mvcc.cc:205] Tried to move back new op lower bound from 7315408024817496064 to 7315408024748507136. Current Snapshot: MvccSnapshot[applied={T|T < 7315408024817496064}]
I20260806 03:51:27.324185 11346 catalog_manager.cc:1347] Generated new cluster ID: e687059fcac84616a2252213a0c854e1
I20260806 03:51:27.324259 11346 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 03:51:27.330958 11189 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.253.252" port: 35087 } attempt: 3
Aug 06 03:51:27 dist-test-slave-6ntk krb5kdc[10991](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988287, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.253.253@KRBTEST.COM for kudu/127.11.253.252@KRBTEST.COM
I20260806 03:51:27.341114 11192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:27.331324 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.252:35087 (local address 127.0.0.1:46704)
0806 03:51:27.331413 (+ 89us) negotiation.cc:107] Waiting for socket to connect
0806 03:51:27.331424 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 03:51:27.331470 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:51:27.331719 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:51:27.331720 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:51:27.331942 (+ 222us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:51:27.331993 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:51:27.331994 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:51:27.333031 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:51:27.333032 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:51:27.333991 (+ 959us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:51:27.333994 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:51:27.334622 (+ 628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:51:27.334623 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:51:27.334687 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:27.335247 (+ 560us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:51:27.335253 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:51:27.337190 (+ 1937us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:51:27.339817 (+ 2627us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:51:27.339818 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:51:27.339819 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:51:27.340075 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:51:27.340348 (+ 273us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:51:27.340348 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:51:27.340349 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:51:27.340423 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:51:27.340767 (+ 344us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:51:27.340768 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:51:27.340831 (+ 63us) client_negotiation.cc:769] Sending connection context
0806 03:51:27.340891 (+ 60us) client_negotiation.cc:240] Negotiation successful
0806 03:51:27.341007 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":68}
I20260806 03:51:27.341434 11301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:27.331389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46704 (local address 127.11.253.252:35087)
0806 03:51:27.331431 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0806 03:51:27.331433 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:51:27.331580 (+ 147us) server_negotiation.cc:407] Connection header received
0806 03:51:27.331607 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:51:27.331608 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:51:27.331620 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:51:27.331668 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:51:27.332076 (+ 408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:27.332920 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:27.334113 (+ 1193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:27.334487 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:27.334556 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:27.337339 (+ 2783us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:51:27.337341 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:51:27.337342 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:51:27.337350 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:51:27.339573 (+ 2223us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:51:27.340182 (+ 609us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:51:27.340184 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:51:27.340184 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:51:27.340261 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:51:27.340507 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:51:27.340508 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:51:27.340509 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:51:27.340660 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:51:27.340722 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0806 03:51:27.341265 (+ 543us) server_negotiation.cc:299] Negotiation successful
0806 03:51:27.341346 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260806 03:51:27.341715 11346 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 03:51:27.343308 11346 catalog_manager.cc:1504] Loading token signing keys...
I20260806 03:51:27.343798 11189 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b201733feb5f4a5c87ebf08707f39461: Bootstrap starting.
I20260806 03:51:27.344594 11189 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b201733feb5f4a5c87ebf08707f39461: Neither blocks nor log segments found. Creating new log.
I20260806 03:51:27.344923 11189 log.cc:824] T 00000000000000000000000000000000 P b201733feb5f4a5c87ebf08707f39461: Log is configured to *not* fsync() on all Append() calls
I20260806 03:51:27.350658 11189 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b201733feb5f4a5c87ebf08707f39461: No bootstrap required, opened a new log
I20260806 03:51:27.352918 11189 raft_consensus.cc:348] T 00000000000000000000000000000000 P b201733feb5f4a5c87ebf08707f39461 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b37dc1a270849ebbed01fe4798c3c01" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 37231 } } peers { permanent_uuid: "b201733feb5f4a5c87ebf08707f39461" member_type: VOTER last_known_addr { host: "127.11.253.253" port: 36387 } } peers { permanent_uuid: "a3b7c28c24d34118a81fbe40b2ff06fb" member_type: VOTER last_known_addr { host: "127.11.253.252" port: 35087 } }
I20260806 03:51:27.353070 11189 raft_consensus.cc:729] T 00000000000000000000000000000000 P b201733feb5f4a5c87ebf08707f39461 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b201733feb5f4a5c87ebf08707f39461, State: Initialized, Role: FOLLOWER
I20260806 03:51:27.353153 11189 consensus_queue.cc:260] T 00000000000000000000000000000000 P b201733feb5f4a5c87ebf08707f39461 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b37dc1a270849ebbed01fe4798c3c01" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 37231 } } peers { permanent_uuid: "b201733feb5f4a5c87ebf08707f39461" member_type: VOTER last_known_addr { host: "127.11.253.253" port: 36387 } } peers { permanent_uuid: "a3b7c28c24d34118a81fbe40b2ff06fb" member_type: VOTER last_known_addr { host: "127.11.253.252" port: 35087 } }
I20260806 03:51:27.353531 11189 sys_catalog.cc:564] T 00000000000000000000000000000000 P b201733feb5f4a5c87ebf08707f39461 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 03:51:27.353986 11158 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3b37dc1a270849ebbed01fe4798c3c01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "b201733feb5f4a5c87ebf08707f39461"
I20260806 03:51:27.354104 11158 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b201733feb5f4a5c87ebf08707f39461 [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:51:27.355041 11158 raft_consensus.cc:2454] T 00000000000000000000000000000000 P b201733feb5f4a5c87ebf08707f39461 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b37dc1a270849ebbed01fe4798c3c01 in term 1.
I20260806 03:51:27.355407 11364 sys_catalog.cc:455] T 00000000000000000000000000000000 P b201733feb5f4a5c87ebf08707f39461 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b37dc1a270849ebbed01fe4798c3c01" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 37231 } } peers { permanent_uuid: "b201733feb5f4a5c87ebf08707f39461" member_type: VOTER last_known_addr { host: "127.11.253.253" port: 36387 } } peers { permanent_uuid: "a3b7c28c24d34118a81fbe40b2ff06fb" member_type: VOTER last_known_addr { host: "127.11.253.252" port: 35087 } } }
I20260806 03:51:27.355497 11364 sys_catalog.cc:458] T 00000000000000000000000000000000 P b201733feb5f4a5c87ebf08707f39461 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 03:51:27.355530 11157 raft_consensus.cc:1261] T 00000000000000000000000000000000 P b201733feb5f4a5c87ebf08707f39461 [term 1 FOLLOWER]: Refusing update from remote peer 3b37dc1a270849ebbed01fe4798c3c01: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 03:51:27.355588 11346 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3b37dc1a270849ebbed01fe4798c3c01: Generated new TSK 0
I20260806 03:51:27.355762 11346 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 03:51:27.356178 11338 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 3b37dc1a270849ebbed01fe4798c3c01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b201733feb5f4a5c87ebf08707f39461" member_type: VOTER last_known_addr { host: "127.11.253.253" port: 36387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 03:51:27.358014 11364 sys_catalog.cc:455] T 00000000000000000000000000000000 P b201733feb5f4a5c87ebf08707f39461 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3b37dc1a270849ebbed01fe4798c3c01. Latest consensus state: current_term: 1 leader_uuid: "3b37dc1a270849ebbed01fe4798c3c01" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b37dc1a270849ebbed01fe4798c3c01" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 37231 } } peers { permanent_uuid: "b201733feb5f4a5c87ebf08707f39461" member_type: VOTER last_known_addr { host: "127.11.253.253" port: 36387 } } peers { permanent_uuid: "a3b7c28c24d34118a81fbe40b2ff06fb" member_type: VOTER last_known_addr { host: "127.11.253.252" port: 35087 } } }
I20260806 03:51:27.358111 11364 sys_catalog.cc:458] T 00000000000000000000000000000000 P b201733feb5f4a5c87ebf08707f39461 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 03:51:27.358171 11364 sys_catalog.cc:455] T 00000000000000000000000000000000 P b201733feb5f4a5c87ebf08707f39461 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "3b37dc1a270849ebbed01fe4798c3c01" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b37dc1a270849ebbed01fe4798c3c01" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 37231 } } peers { permanent_uuid: "b201733feb5f4a5c87ebf08707f39461" member_type: VOTER last_known_addr { host: "127.11.253.253" port: 36387 } } peers { permanent_uuid: "a3b7c28c24d34118a81fbe40b2ff06fb" member_type: VOTER last_known_addr { host: "127.11.253.252" port: 35087 } } }
I20260806 03:51:27.358240 11364 sys_catalog.cc:458] T 00000000000000000000000000000000 P b201733feb5f4a5c87ebf08707f39461 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 03:51:27.360021 11388 catalog_manager.cc:1259] Loaded cluster ID: e687059fcac84616a2252213a0c854e1
I20260806 03:51:27.360100 11388 catalog_manager.cc:1552] T 00000000000000000000000000000000 P b201733feb5f4a5c87ebf08707f39461: loading cluster ID for follower catalog manager: success
I20260806 03:51:27.361261 11388 catalog_manager.cc:1574] T 00000000000000000000000000000000 P b201733feb5f4a5c87ebf08707f39461: acquiring CA information for follower catalog manager: success
I20260806 03:51:27.361646 11388 catalog_manager.cc:1602] T 00000000000000000000000000000000 P b201733feb5f4a5c87ebf08707f39461: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260806 03:51:27.383718 11194 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 03:51:27.382147 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51330 (local address 127.11.253.253:36387)
0806 03:51:27.382198 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0806 03:51:27.382200 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:51:27.382949 (+ 749us) server_negotiation.cc:407] Connection header received
0806 03:51:27.383067 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:51:27.383068 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:51:27.383081 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:51:27.383115 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:51:27.383639 (+ 524us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51330: BlockingRecv error: recv got EOF from 127.0.0.1:51330 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
W20260806 03:51:27.384793 11097 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 03:51:27.382611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52946 (local address 127.11.253.254:37231)
0806 03:51:27.382654 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0806 03:51:27.382656 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:51:27.384163 (+ 1507us) server_negotiation.cc:407] Connection header received
0806 03:51:27.384300 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:51:27.384317 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:51:27.384333 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:51:27.384372 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:51:27.384725 (+ 353us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52946: BlockingRecv error: recv got EOF from 127.0.0.1:52946 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260806 03:51:27.386051 11301 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 03:51:27.385269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46708 (local address 127.11.253.252:35087)
0806 03:51:27.385325 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0806 03:51:27.385327 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:51:27.385450 (+ 123us) server_negotiation.cc:407] Connection header received
0806 03:51:27.385550 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:51:27.385551 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:51:27.385561 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:51:27.385593 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:51:27.385983 (+ 390us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46708: BlockingRecv error: recv got EOF from 127.0.0.1:46708 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 03:51:27.392315 11194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:27.387730 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51332 (local address 127.11.253.253:36387)
0806 03:51:27.387766 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0806 03:51:27.387768 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:51:27.388060 (+ 292us) server_negotiation.cc:407] Connection header received
0806 03:51:27.388249 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:51:27.388250 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:51:27.388294 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:51:27.388354 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 03:51:27.388571 (+ 217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:27.389518 (+ 947us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:27.391117 (+ 1599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:27.391472 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:27.391517 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:27.391889 (+ 372us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 03:51:27.392026 (+ 137us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 03:51:27.392080 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0806 03:51:27.392205 (+ 125us) server_negotiation.cc:299] Negotiation successful
0806 03:51:27.392226 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
W20260806 03:51:27.395974 11301 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 03:51:27.395160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46710 (local address 127.11.253.252:35087)
0806 03:51:27.395203 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0806 03:51:27.395206 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 03:51:27.395402 (+ 196us) server_negotiation.cc:407] Connection header received
0806 03:51:27.395521 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:51:27.395522 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:51:27.395541 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:51:27.395573 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:51:27.395926 (+ 353us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46710: BlockingRecv error: recv got EOF from 127.0.0.1:46710 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 03:51:27.401921 11194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:27.398362 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51336 (local address 127.11.253.253:36387)
0806 03:51:27.398399 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0806 03:51:27.398401 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:51:27.398666 (+ 265us) server_negotiation.cc:407] Connection header received
0806 03:51:27.398766 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:51:27.398767 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:51:27.398780 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:51:27.398811 (+ 31us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 03:51:27.398989 (+ 178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:27.399804 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:27.401031 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:27.401368 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:27.401411 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:27.401623 (+ 212us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 03:51:27.401658 (+ 35us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 03:51:27.401688 (+ 30us) server_negotiation.cc:1035] Waiting for connection context
0806 03:51:27.401800 (+ 112us) server_negotiation.cc:299] Negotiation successful
0806 03:51:27.401823 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
W20260806 03:51:27.405385 11301 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 03:51:27.404577 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46720 (local address 127.11.253.252:35087)
0806 03:51:27.404619 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0806 03:51:27.404621 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:51:27.404809 (+ 188us) server_negotiation.cc:407] Connection header received
0806 03:51:27.404915 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:51:27.404916 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:51:27.404928 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:51:27.404962 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:51:27.405331 (+ 369us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46720: BlockingRecv error: recv got EOF from 127.0.0.1:46720 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 03:51:27.412153 11194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:27.408633 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51344 (local address 127.11.253.253:36387)
0806 03:51:27.408689 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0806 03:51:27.408690 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 03:51:27.408949 (+ 259us) server_negotiation.cc:407] Connection header received
0806 03:51:27.409051 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:51:27.409052 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:51:27.409062 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:51:27.409092 (+ 30us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 03:51:27.409265 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:27.410102 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:27.411299 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:27.411637 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:27.411678 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:27.411881 (+ 203us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 03:51:27.411917 (+ 36us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 03:51:27.411945 (+ 28us) server_negotiation.cc:1035] Waiting for connection context
0806 03:51:27.412052 (+ 107us) server_negotiation.cc:299] Negotiation successful
0806 03:51:27.412069 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
W20260806 03:51:27.415575 11301 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 03:51:27.414747 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46722 (local address 127.11.253.252:35087)
0806 03:51:27.414788 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0806 03:51:27.414790 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:51:27.414976 (+ 186us) server_negotiation.cc:407] Connection header received
0806 03:51:27.415074 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:51:27.415075 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:51:27.415086 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:51:27.415116 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:51:27.415518 (+ 402us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46722: BlockingRecv error: recv got EOF from 127.0.0.1:46722 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
I20260806 03:51:27.424780 11194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:27.421132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51350 (local address 127.11.253.253:36387)
0806 03:51:27.421174 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0806 03:51:27.421176 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:51:27.421497 (+ 321us) server_negotiation.cc:407] Connection header received
0806 03:51:27.421613 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:51:27.421614 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:51:27.421627 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:51:27.421659 (+ 32us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 03:51:27.421859 (+ 200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:27.422681 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:27.423876 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:27.424186 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:27.424236 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:27.424488 (+ 252us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 03:51:27.424525 (+ 37us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 03:51:27.424559 (+ 34us) server_negotiation.cc:1035] Waiting for connection context
0806 03:51:27.424681 (+ 122us) server_negotiation.cc:299] Negotiation successful
0806 03:51:27.424701 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
W20260806 03:51:27.428258 11301 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 03:51:27.427450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46724 (local address 127.11.253.252:35087)
0806 03:51:27.427492 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0806 03:51:27.427495 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 03:51:27.427684 (+ 189us) server_negotiation.cc:407] Connection header received
0806 03:51:27.427790 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:51:27.427791 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:51:27.427804 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:51:27.427838 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:51:27.428202 (+ 364us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46724: BlockingRecv error: recv got EOF from 127.0.0.1:46724 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
I20260806 03:51:27.441766 11194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:27.437668 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51354 (local address 127.11.253.253:36387)
0806 03:51:27.437715 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0806 03:51:27.437717 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:51:27.438067 (+ 350us) server_negotiation.cc:407] Connection header received
0806 03:51:27.438239 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:51:27.438241 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:51:27.438254 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:51:27.438288 (+ 34us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 03:51:27.438596 (+ 308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:27.439457 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:27.440810 (+ 1353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:27.441122 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:27.441173 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:27.441466 (+ 293us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 03:51:27.441506 (+ 40us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 03:51:27.441539 (+ 33us) server_negotiation.cc:1035] Waiting for connection context
0806 03:51:27.441660 (+ 121us) server_negotiation.cc:299] Negotiation successful
0806 03:51:27.441682 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
W20260806 03:51:27.445459 11301 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 03:51:27.444558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46736 (local address 127.11.253.252:35087)
0806 03:51:27.444603 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0806 03:51:27.444605 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:51:27.444802 (+ 197us) server_negotiation.cc:407] Connection header received
0806 03:51:27.444912 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:51:27.444913 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:51:27.444927 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:51:27.444959 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:51:27.445393 (+ 434us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46736: BlockingRecv error: recv got EOF from 127.0.0.1:46736 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 03:51:27.468703 11194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:27.464653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51368 (local address 127.11.253.253:36387)
0806 03:51:27.464699 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0806 03:51:27.464722 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0806 03:51:27.465117 (+ 395us) server_negotiation.cc:407] Connection header received
0806 03:51:27.465298 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:51:27.465299 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:51:27.465312 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:51:27.465345 (+ 33us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 03:51:27.465699 (+ 354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:27.466708 (+ 1009us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:27.467846 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:27.468125 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:27.468174 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:27.468397 (+ 223us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 03:51:27.468448 (+ 51us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 03:51:27.468480 (+ 32us) server_negotiation.cc:1035] Waiting for connection context
0806 03:51:27.468599 (+ 119us) server_negotiation.cc:299] Negotiation successful
0806 03:51:27.468621 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
W20260806 03:51:27.480396 11301 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 03:51:27.476984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46750 (local address 127.11.253.252:35087)
0806 03:51:27.477036 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0806 03:51:27.477038 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:51:27.479653 (+ 2615us) server_negotiation.cc:407] Connection header received
0806 03:51:27.479790 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:51:27.479791 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:51:27.479806 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:51:27.479838 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:51:27.480329 (+ 491us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46750: BlockingRecv error: recv got EOF from 127.0.0.1:46750 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
I20260806 03:51:27.517413 11194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:27.513995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51382 (local address 127.11.253.253:36387)
0806 03:51:27.514057 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0806 03:51:27.514059 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:51:27.514224 (+ 165us) server_negotiation.cc:407] Connection header received
0806 03:51:27.514286 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:51:27.514287 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:51:27.514299 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:51:27.514380 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 03:51:27.514573 (+ 193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:27.515313 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:27.516542 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:27.516862 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:27.516937 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:27.517112 (+ 175us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 03:51:27.517150 (+ 38us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 03:51:27.517207 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0806 03:51:27.517287 (+ 80us) server_negotiation.cc:299] Negotiation successful
0806 03:51:27.517307 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
W20260806 03:51:27.520053 11301 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 03:51:27.519463 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46762 (local address 127.11.253.252:35087)
0806 03:51:27.519503 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0806 03:51:27.519505 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:51:27.519513 (+ 8us) server_negotiation.cc:407] Connection header received
0806 03:51:27.519538 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:51:27.519539 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:51:27.519550 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:51:27.519609 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:51:27.519991 (+ 382us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46762: BlockingRecv error: recv got EOF from 127.0.0.1:46762 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
I20260806 03:51:27.587603 11194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:27.585343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51394 (local address 127.11.253.253:36387)
0806 03:51:27.585435 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0806 03:51:27.585437 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:51:27.585534 (+ 97us) server_negotiation.cc:407] Connection header received
0806 03:51:27.585583 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:51:27.585584 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:51:27.585594 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:51:27.585658 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 03:51:27.585780 (+ 122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:27.586297 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:27.587040 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:27.587219 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:27.587270 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:27.587408 (+ 138us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 03:51:27.587435 (+ 27us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 03:51:27.587473 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0806 03:51:27.587525 (+ 52us) server_negotiation.cc:299] Negotiation successful
0806 03:51:27.587540 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
W20260806 03:51:27.590374 11301 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 03:51:27.589596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46776 (local address 127.11.253.252:35087)
0806 03:51:27.589666 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0806 03:51:27.589668 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:51:27.589719 (+ 51us) server_negotiation.cc:407] Connection header received
0806 03:51:27.589791 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:51:27.589792 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:51:27.589804 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:51:27.589849 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:51:27.590274 (+ 425us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46776: BlockingRecv error: recv got EOF from 127.0.0.1:46776 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
I20260806 03:51:27.722049 11194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:27.719764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51410 (local address 127.11.253.253:36387)
0806 03:51:27.719838 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0806 03:51:27.719840 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:51:27.719880 (+ 40us) server_negotiation.cc:407] Connection header received
0806 03:51:27.719959 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:51:27.719960 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:51:27.719969 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:51:27.720022 (+ 53us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 03:51:27.720190 (+ 168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:27.720677 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:27.721416 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:27.721589 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:27.721639 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:27.721766 (+ 127us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 03:51:27.721793 (+ 27us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 03:51:27.721831 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0806 03:51:27.721918 (+ 87us) server_negotiation.cc:299] Negotiation successful
0806 03:51:27.721933 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
W20260806 03:51:27.724758 11301 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 03:51:27.724041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46792 (local address 127.11.253.252:35087)
0806 03:51:27.724102 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0806 03:51:27.724104 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:51:27.724202 (+ 98us) server_negotiation.cc:407] Connection header received
0806 03:51:27.724272 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:51:27.724274 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:51:27.724287 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:51:27.724360 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:51:27.724697 (+ 337us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46792: BlockingRecv error: recv got EOF from 127.0.0.1:46792 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
I20260806 03:51:27.872659 11515 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3b37dc1a270849ebbed01fe4798c3c01 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "3b37dc1a270849ebbed01fe4798c3c01" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b37dc1a270849ebbed01fe4798c3c01" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 37231 } } peers { permanent_uuid: "b201733feb5f4a5c87ebf08707f39461" member_type: VOTER last_known_addr { host: "127.11.253.253" port: 36387 } } peers { permanent_uuid: "a3b7c28c24d34118a81fbe40b2ff06fb" member_type: VOTER last_known_addr { host: "127.11.253.252" port: 35087 } } }
I20260806 03:51:27.872826 11515 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3b37dc1a270849ebbed01fe4798c3c01 [sys.catalog]: This master's current role is: LEADER
I20260806 03:51:27.984762 11194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:27.982241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51412 (local address 127.11.253.253:36387)
0806 03:51:27.982321 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0806 03:51:27.982323 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:51:27.982366 (+ 43us) server_negotiation.cc:407] Connection header received
0806 03:51:27.982479 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:51:27.982481 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:51:27.982493 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:51:27.982531 (+ 38us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 03:51:27.982760 (+ 229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:27.983272 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:27.984170 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:27.984358 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:27.984421 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:27.984556 (+ 135us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 03:51:27.984585 (+ 29us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 03:51:27.984624 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0806 03:51:27.984681 (+ 57us) server_negotiation.cc:299] Negotiation successful
0806 03:51:27.984698 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
W20260806 03:51:27.987429 11301 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 03:51:27.986835 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46796 (local address 127.11.253.252:35087)
0806 03:51:27.986920 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0806 03:51:27.986922 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:51:27.986930 (+ 8us) server_negotiation.cc:407] Connection header received
0806 03:51:27.986991 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:51:27.986992 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:51:27.987003 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:51:27.987051 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:51:27.987377 (+ 326us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46796: BlockingRecv error: recv got EOF from 127.0.0.1:46796 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
I20260806 03:51:28.299306 11337 catalog_manager.cc:1259] Loaded cluster ID: e687059fcac84616a2252213a0c854e1
I20260806 03:51:28.299394 11337 catalog_manager.cc:1552] T 00000000000000000000000000000000 P a3b7c28c24d34118a81fbe40b2ff06fb: loading cluster ID for follower catalog manager: success
I20260806 03:51:28.300203 11337 catalog_manager.cc:1574] T 00000000000000000000000000000000 P a3b7c28c24d34118a81fbe40b2ff06fb: acquiring CA information for follower catalog manager: success
I20260806 03:51:28.300468 11337 catalog_manager.cc:1602] T 00000000000000000000000000000000 P a3b7c28c24d34118a81fbe40b2ff06fb: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 03:51:28.503161 11546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:28.500647 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51422 (local address 127.11.253.253:36387)
0806 03:51:28.500795 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0806 03:51:28.500798 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 03:51:28.500812 (+ 14us) server_negotiation.cc:407] Connection header received
0806 03:51:28.500847 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:51:28.500848 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:51:28.500864 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:51:28.500959 (+ 95us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 03:51:28.501133 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:28.501690 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:28.502495 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:28.502687 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:28.502744 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:28.502866 (+ 122us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 03:51:28.502895 (+ 29us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 03:51:28.502935 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0806 03:51:28.502994 (+ 59us) server_negotiation.cc:299] Negotiation successful
0806 03:51:28.503013 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":68,"threads_started":1}
I20260806 03:51:28.507566 11558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:28.504981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46802 (local address 127.11.253.252:35087)
0806 03:51:28.505087 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0806 03:51:28.505089 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:51:28.505098 (+ 9us) server_negotiation.cc:407] Connection header received
0806 03:51:28.505123 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:51:28.505123 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:51:28.505177 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:51:28.505246 (+ 69us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 03:51:28.505373 (+ 127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:28.505844 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:28.506670 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:28.506907 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:28.506987 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:28.507098 (+ 111us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 03:51:28.507275 (+ 177us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 03:51:28.507362 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0806 03:51:28.507393 (+ 31us) server_negotiation.cc:299] Negotiation successful
0806 03:51:28.507416 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79,"thread_start_us":57,"threads_started":1}
I20260806 03:51:28.512188 11558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:28.509488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46806 (local address 127.11.253.252:35087)
0806 03:51:28.509533 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0806 03:51:28.509534 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 03:51:28.509545 (+ 11us) server_negotiation.cc:407] Connection header received
0806 03:51:28.509573 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:51:28.509574 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:51:28.509585 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:51:28.509651 (+ 66us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 03:51:28.509839 (+ 188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:28.510565 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:28.511470 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:28.511676 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:28.511748 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:28.511908 (+ 160us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 03:51:28.511947 (+ 39us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 03:51:28.512003 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0806 03:51:28.512070 (+ 67us) server_negotiation.cc:299] Negotiation successful
0806 03:51:28.512090 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260806 03:51:28.512840 11546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:28.509304 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51430 (local address 127.11.253.253:36387)
0806 03:51:28.509346 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0806 03:51:28.509348 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:51:28.509446 (+ 98us) server_negotiation.cc:407] Connection header received
0806 03:51:28.509553 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:51:28.509554 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:51:28.509567 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:51:28.509623 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 03:51:28.509791 (+ 168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:28.510530 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:28.511847 (+ 1317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:28.512171 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:28.512257 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:28.512499 (+ 242us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 03:51:28.512558 (+ 59us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 03:51:28.512611 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0806 03:51:28.512705 (+ 94us) server_negotiation.cc:299] Negotiation successful
0806 03:51:28.512734 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260806 03:51:28.513160 11565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:28.509288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52950 (local address 127.11.253.254:37231)
0806 03:51:28.509540 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0806 03:51:28.509543 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 03:51:28.509712 (+ 169us) server_negotiation.cc:407] Connection header received
0806 03:51:28.509791 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:51:28.509792 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:51:28.509850 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:51:28.509952 (+ 102us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 03:51:28.510108 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:28.510855 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:28.511992 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:28.512357 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:28.512428 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:28.512709 (+ 281us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 03:51:28.512839 (+ 130us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 03:51:28.512943 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0806 03:51:28.513011 (+ 68us) server_negotiation.cc:299] Negotiation successful
0806 03:51:28.513038 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":227,"thread_start_us":67,"threads_started":1}
I20260806 03:51:28.519090 12279 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskToVWRe/build/release/bin/kudu with pid 11009
I20260806 03:51:28.531337 12279 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskToVWRe/build/release/bin/kudu with pid 11101
I20260806 03:51:28.542819 12279 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskToVWRe/build/release/bin/kudu with pid 11198
2026-08-06T03:51:28Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (3088 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785988004637995-12279-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 03:51:28 dist-test-slave-6ntk krb5kdc[11570](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 03:51:28 dist-test-slave-6ntk krb5kdc[11570](info): set up 2 sockets
Aug 06 03:51:28 dist-test-slave-6ntk krb5kdc[11570](info): commencing operation
krb5kdc: starting...
W20260806 03:51:29.798606 12279 mini_kdc.cc:121] Time spent starting KDC: real 1.229s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785988004637995-12279-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785988004637995-12279-0/krb5kdc/test-user.keytab.
Aug 06 03:51:29 dist-test-slave-6ntk krb5kdc[11570](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988289, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T03:51:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T03:51:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.253.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.253.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.253.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.254.keytab.
Entry for principal kudu/127.11.253.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.254.keytab.
I20260806 03:51:29.929507 12279 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskToVWRe/build/release/bin/kudu
/tmp/dist-test-taskToVWRe/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785988004637995-12279-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785988004637995-12279-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785988004637995-12279-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785988004637995-12279-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.253.254:35715
--webserver_interface=127.11.253.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.254.keytab
--principal=kudu/127.11.253.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.253.212:41027
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.253.254:35715,127.11.253.253:39559,127.11.253.252:35693
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785988004637995-12279-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785988004637995-12279-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785988004637995-12279-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:51:30.013985 11592 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 03:51:30.014248 11592 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:51:30.014279 11592 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:51:30.014324 11592 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:51:30.015520 11592 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 03:51:30.015594 11592 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:51:30.015617 11592 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:51:30.015638 11592 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 03:51:30.015671 11592 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 03:51:30.017194 11592 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.253.212:41027
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785988004637995-12279-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785988004637995-12279-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.253.254:35715,127.11.253.253:39559,127.11.253.252:35693
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.254.keytab
--principal=kudu/127.11.253.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.253.254:35715
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785988004637995-12279-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.253.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11592
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785988004637995-12279-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:51:30.017493 11592 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:51:30.017735 11592 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:51:30.022671 11601 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:51:30.022671 11598 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:51:30.022691 11599 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:51:30.023180 11592 server_base.cc:1034] running on GCE node
I20260806 03:51:30.023455 11592 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:51:30.023764 11592 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:51:30.024950 11592 hybrid_clock.cc:648] HybridClock initialized: now 1785988290024914 us; error 59 us; skew 500 ppm
Aug 06 03:51:30 dist-test-slave-6ntk krb5kdc[11570](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988290, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.253.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:51:30.027709 11592 init.cc:377] Logged in from keytab as kudu/127.11.253.254@KRBTEST.COM (short username kudu)
I20260806 03:51:30.028225 11592 webserver.cc:460] Webserver started at http://127.11.253.254:41825/ using document root <none> and password file <none>
I20260806 03:51:30.028465 11592 fs_manager.cc:359] Metadata directory not provided
I20260806 03:51:30.028537 11592 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:51:30.028693 11592 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:51:30.029939 11592 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785988004637995-12279-0/minicluster-data/master-0/data/instance:
uuid: "dbdeb40a549e4c27ba6f21fa8f59c37c"
format_stamp: "Formatted at 2026-08-06 03:51:30 on dist-test-slave-6ntk"
server_key: "11ee67faabb59c1ae4b19c2266d4e268"
server_key_iv: "416a24109d39cf2192fd19609a012625"
server_key_version: "encryptionkey@0"
I20260806 03:51:30.030346 11592 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785988004637995-12279-0/minicluster-data/master-0/wal/instance:
uuid: "dbdeb40a549e4c27ba6f21fa8f59c37c"
format_stamp: "Formatted at 2026-08-06 03:51:30 on dist-test-slave-6ntk"
server_key: "11ee67faabb59c1ae4b19c2266d4e268"
server_key_iv: "416a24109d39cf2192fd19609a012625"
server_key_version: "encryptionkey@0"
I20260806 03:51:30.031791 11592 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 03:51:30.032828 11610 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:51:30.033083 11592 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 03:51:30.033202 11592 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785988004637995-12279-0/minicluster-data/master-0/data,/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785988004637995-12279-0/minicluster-data/master-0/wal
uuid: "dbdeb40a549e4c27ba6f21fa8f59c37c"
format_stamp: "Formatted at 2026-08-06 03:51:30 on dist-test-slave-6ntk"
server_key: "11ee67faabb59c1ae4b19c2266d4e268"
server_key_iv: "416a24109d39cf2192fd19609a012625"
server_key_version: "encryptionkey@0"
I20260806 03:51:30.033308 11592 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785988004637995-12279-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785988004637995-12279-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785988004637995-12279-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:51:30.042790 11592 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:51:30.045022 11592 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:51:30.045217 11592 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:51:30.049675 11592 rpc_server.cc:307] RPC server started. Bound to: 127.11.253.254:35715
I20260806 03:51:30.049732 11682 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.253.254:35715 every 8 connection(s)
I20260806 03:51:30.050180 11592 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785988004637995-12279-0/minicluster-data/master-0/data/info.pb
I20260806 03:51:30.051127 11683 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:51:30.052830 11683 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.253.254" port: 35715 } has no permanent_uuid. Determining permanent_uuid...
I20260806 03:51:30.054921 12279 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskToVWRe/build/release/bin/kudu as pid 11592
I20260806 03:51:30.055006 12279 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785988004637995-12279-0/minicluster-data/master-0/wal/instance
I20260806 03:51:30.055212 12279 external_mini_cluster.cc:1428] Setting key 3bc44dd0819fb630ce9bb6084cfec842
Aug 06 03:51:30 dist-test-slave-6ntk krb5kdc[11570](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988290, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.253.254@KRBTEST.COM for kudu/127.11.253.254@KRBTEST.COM
I20260806 03:51:30.063768 11687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:30.053544 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:35715 (local address 127.0.0.1:43050)
0806 03:51:30.053711 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0806 03:51:30.053749 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0806 03:51:30.054325 (+ 576us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:51:30.054613 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:51:30.054615 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:51:30.054903 (+ 288us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:51:30.055140 (+ 237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:51:30.055146 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:51:30.056320 (+ 1174us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:51:30.056321 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:51:30.057265 (+ 944us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:51:30.057269 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:51:30.057811 (+ 542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:51:30.057812 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:51:30.057924 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:30.058568 (+ 644us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:51:30.058579 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:51:30.060269 (+ 1690us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:51:30.062171 (+ 1902us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:51:30.062178 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:51:30.062180 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:51:30.062386 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:51:30.062655 (+ 269us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:51:30.062655 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:51:30.062656 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:51:30.062749 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:51:30.063136 (+ 387us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:51:30.063142 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:51:30.063207 (+ 65us) client_negotiation.cc:769] Sending connection context
0806 03:51:30.063320 (+ 113us) client_negotiation.cc:240] Negotiation successful
0806 03:51:30.063577 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"mutex_wait_us":88,"thread_start_us":71,"threads_started":1}
I20260806 03:51:30.063771 11688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:30.053701 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43050 (local address 127.11.253.254:35715)
0806 03:51:30.053830 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0806 03:51:30.053835 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 03:51:30.053925 (+ 90us) server_negotiation.cc:407] Connection header received
0806 03:51:30.054479 (+ 554us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:51:30.054481 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:51:30.054498 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:51:30.054575 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:51:30.055315 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:30.056177 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:30.057376 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:30.057713 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:30.057792 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:30.060380 (+ 2588us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:51:30.060390 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:51:30.060390 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:51:30.060402 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:51:30.062075 (+ 1673us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:51:30.062493 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:51:30.062496 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:51:30.062496 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:51:30.062582 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:51:30.062826 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:51:30.062826 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:51:30.062827 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:51:30.063056 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:51:30.063158 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0806 03:51:30.063355 (+ 197us) server_negotiation.cc:299] Negotiation successful
0806 03:51:30.063501 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98,"thread_start_us":69,"threads_started":1}
I20260806 03:51:30.064635 11683 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.253.253" port: 39559 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.11.253.253@KRBTEST.COM; defaulting to no policy
I20260806 03:51:30.065024 11687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:30.064883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.253:39559 (local address 127.0.0.1:52002)
0806 03:51:30.064949 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0806 03:51:30.064997 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.253.253:39559: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260806 03:51:30.065174 11611 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.11.253.253:39559: connect: Connection refused (error 111)
I20260806 03:51:30.065608 11687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:30.065508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.253:39559 (local address 127.0.0.1:52008)
0806 03:51:30.065559 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0806 03:51:30.065584 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.253.253:39559: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260806 03:51:30.065721 11683 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.253.253:39559: Network error: Client connection negotiation failed: client connection to 127.11.253.253:39559: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.253.253@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.253.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.253.keytab.
Entry for principal kudu/127.11.253.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.253.keytab.
I20260806 03:51:30.085059 12279 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskToVWRe/build/release/bin/kudu
/tmp/dist-test-taskToVWRe/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785988004637995-12279-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785988004637995-12279-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785988004637995-12279-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785988004637995-12279-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.253.253:39559
--webserver_interface=127.11.253.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.253.keytab
--principal=kudu/127.11.253.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.253.212:41027
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.253.254:35715,127.11.253.253:39559,127.11.253.252:35693
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785988004637995-12279-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785988004637995-12279-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785988004637995-12279-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 03:51:30.114933 11683 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.253.253" port: 39559 } attempt: 1
I20260806 03:51:30.115437 11687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:30.115294 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.253:39559 (local address 127.0.0.1:52020)
0806 03:51:30.115360 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0806 03:51:30.115406 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.253.253:39559: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260806 03:51:30.115839 11687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:30.115731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.253:39559 (local address 127.0.0.1:52026)
0806 03:51:30.115782 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0806 03:51:30.115808 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.253.253:39559: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260806 03:51:30.115928 11683 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.253.253:39559: Network error: Client connection negotiation failed: client connection to 127.11.253.253:39559: connect: Connection refused (error 111)
W20260806 03:51:30.171846 11693 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 03:51:30.172106 11693 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:51:30.172138 11693 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:51:30.172180 11693 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:51:30.173349 11693 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 03:51:30.173419 11693 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:51:30.173442 11693 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:51:30.173462 11693 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 03:51:30.173494 11693 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 03:51:30.175064 11693 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.253.212:41027
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785988004637995-12279-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785988004637995-12279-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.11.253.254:35715,127.11.253.253:39559,127.11.253.252:35693
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.253.keytab
--principal=kudu/127.11.253.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.253.253:39559
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785988004637995-12279-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.11.253.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11693
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785988004637995-12279-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:51:30.175433 11693 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:51:30.175689 11693 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:51:30.181318 11703 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:51:30.181422 11699 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:51:30.181329 11706 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:51:30.181612 11693 server_base.cc:1034] running on GCE node
I20260806 03:51:30.181939 11693 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:51:30.182242 11693 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:51:30.183456 11693 hybrid_clock.cc:648] HybridClock initialized: now 1785988290183452 us; error 39 us; skew 500 ppm
I20260806 03:51:30.184073 11683 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.253.253" port: 39559 } attempt: 2
I20260806 03:51:30.184473 11687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:30.184329 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.253:39559 (local address 127.0.0.1:52028)
0806 03:51:30.184389 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0806 03:51:30.184445 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.253.253:39559: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260806 03:51:30.185704 11687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:30.184898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.253:39559 (local address 127.0.0.1:52040)
0806 03:51:30.185637 (+ 739us) negotiation.cc:107] Waiting for socket to connect
0806 03:51:30.185671 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.253.253:39559: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":720}
W20260806 03:51:30.185887 11683 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.253.253:39559: Network error: Client connection negotiation failed: client connection to 127.11.253.253:39559: connect: Connection refused (error 111)
Aug 06 03:51:30 dist-test-slave-6ntk krb5kdc[11570](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988290, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.253.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:51:30.187049 11693 init.cc:377] Logged in from keytab as kudu/127.11.253.253@KRBTEST.COM (short username kudu)
I20260806 03:51:30.187687 11693 webserver.cc:460] Webserver started at http://127.11.253.253:43147/ using document root <none> and password file <none>
I20260806 03:51:30.187973 11693 fs_manager.cc:359] Metadata directory not provided
I20260806 03:51:30.188095 11693 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:51:30.188262 11693 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:51:30.189911 11693 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785988004637995-12279-0/minicluster-data/master-1/data/instance:
uuid: "9f42bb2328ab469ea6b81aa84464da5e"
format_stamp: "Formatted at 2026-08-06 03:51:30 on dist-test-slave-6ntk"
server_key: "ffb1eba726a1e93a49ce3e1c89baf0c5"
server_key_iv: "96d1e12af06b4bf1736ece6bc1866b1e"
server_key_version: "encryptionkey@0"
I20260806 03:51:30.190429 11693 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785988004637995-12279-0/minicluster-data/master-1/wal/instance:
uuid: "9f42bb2328ab469ea6b81aa84464da5e"
format_stamp: "Formatted at 2026-08-06 03:51:30 on dist-test-slave-6ntk"
server_key: "ffb1eba726a1e93a49ce3e1c89baf0c5"
server_key_iv: "96d1e12af06b4bf1736ece6bc1866b1e"
server_key_version: "encryptionkey@0"
I20260806 03:51:30.192313 11693 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260806 03:51:30.193384 11718 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:51:30.193735 11693 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 03:51:30.193909 11693 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785988004637995-12279-0/minicluster-data/master-1/data,/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785988004637995-12279-0/minicluster-data/master-1/wal
uuid: "9f42bb2328ab469ea6b81aa84464da5e"
format_stamp: "Formatted at 2026-08-06 03:51:30 on dist-test-slave-6ntk"
server_key: "ffb1eba726a1e93a49ce3e1c89baf0c5"
server_key_iv: "96d1e12af06b4bf1736ece6bc1866b1e"
server_key_version: "encryptionkey@0"
I20260806 03:51:30.193996 11693 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785988004637995-12279-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785988004637995-12279-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785988004637995-12279-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:51:30.209738 11693 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:51:30.212271 11693 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:51:30.212409 11693 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:51:30.216637 11693 rpc_server.cc:307] RPC server started. Bound to: 127.11.253.253:39559
I20260806 03:51:30.216704 11786 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.253.253:39559 every 8 connection(s)
I20260806 03:51:30.217069 11693 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785988004637995-12279-0/minicluster-data/master-1/data/info.pb
I20260806 03:51:30.218071 11787 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:51:30.219651 11787 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.253.254" port: 35715 } has no permanent_uuid. Determining permanent_uuid...
I20260806 03:51:30.221553 12279 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskToVWRe/build/release/bin/kudu as pid 11693
I20260806 03:51:30.221657 12279 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785988004637995-12279-0/minicluster-data/master-1/wal/instance
I20260806 03:51:30.221853 12279 external_mini_cluster.cc:1428] Setting key d59bc18d0c8bc31063e41436a390daef
Aug 06 03:51:30 dist-test-slave-6ntk krb5kdc[11570](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988290, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.253.253@KRBTEST.COM for kudu/127.11.253.254@KRBTEST.COM
I20260806 03:51:30.230077 11688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:30.220405 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43056 (local address 127.11.253.254:35715)
0806 03:51:30.220465 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0806 03:51:30.220467 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:51:30.220677 (+ 210us) server_negotiation.cc:407] Connection header received
0806 03:51:30.221134 (+ 457us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:51:30.221135 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:51:30.221147 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:51:30.221197 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:51:30.222091 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:30.222579 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:30.224032 (+ 1453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:30.224365 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:30.224444 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:30.226916 (+ 2472us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:51:30.226917 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:51:30.226918 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:51:30.226925 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:51:30.228670 (+ 1745us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:51:30.229063 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:51:30.229064 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:51:30.229065 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:51:30.229133 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:51:30.229371 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:51:30.229372 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:51:30.229372 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:51:30.229523 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:51:30.229600 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0806 03:51:30.229838 (+ 238us) server_negotiation.cc:299] Negotiation successful
0806 03:51:30.229969 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260806 03:51:30.230160 11789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:30.220350 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:35715 (local address 127.0.0.1:43056)
0806 03:51:30.220554 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0806 03:51:30.220588 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0806 03:51:30.221048 (+ 460us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:51:30.221255 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:51:30.221258 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:51:30.221655 (+ 397us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:51:30.221992 (+ 337us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:51:30.222000 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:51:30.222682 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:51:30.222683 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:51:30.223936 (+ 1253us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:51:30.223939 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:51:30.224454 (+ 515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:51:30.224455 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:51:30.224556 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:30.225118 (+ 562us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:51:30.225133 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:51:30.226818 (+ 1685us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:51:30.228749 (+ 1931us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:51:30.228756 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:51:30.228757 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:51:30.228986 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:51:30.229209 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:51:30.229209 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:51:30.229210 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:51:30.229306 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:51:30.229595 (+ 289us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:51:30.229600 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:51:30.229706 (+ 106us) client_negotiation.cc:769] Sending connection context
0806 03:51:30.229787 (+ 81us) client_negotiation.cc:240] Negotiation successful
0806 03:51:30.229972 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":89,"threads_started":1}
I20260806 03:51:30.230756 11787 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.253.253" port: 39559 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.11.253.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.253.252@KRBTEST.COM" created.
Aug 06 03:51:30 dist-test-slave-6ntk krb5kdc[11570](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988290, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.253.253@KRBTEST.COM for kudu/127.11.253.253@KRBTEST.COM
I20260806 03:51:30.238890 11791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:30.231136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52048 (local address 127.11.253.253:39559)
0806 03:51:30.231302 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0806 03:51:30.231306 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 03:51:30.231323 (+ 17us) server_negotiation.cc:407] Connection header received
0806 03:51:30.231339 (+ 16us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:51:30.231340 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:51:30.231350 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:51:30.231450 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:51:30.231767 (+ 317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:30.232422 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:30.233200 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:30.233500 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:30.233592 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:30.235830 (+ 2238us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:51:30.235839 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:51:30.235840 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:51:30.235848 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:51:30.237574 (+ 1726us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:51:30.238021 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:51:30.238023 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:51:30.238024 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:51:30.238079 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:51:30.238292 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:51:30.238293 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:51:30.238293 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:51:30.238429 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:51:30.238491 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0806 03:51:30.238655 (+ 164us) server_negotiation.cc:299] Negotiation successful
0806 03:51:30.238790 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":71,"server-negotiator.queue_time_us":141,"thread_start_us":90,"threads_started":1}
I20260806 03:51:30.238962 11789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:30.231059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.253:39559 (local address 127.0.0.1:52048)
0806 03:51:30.231122 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0806 03:51:30.231131 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 03:51:30.231171 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:51:30.231450 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:51:30.231451 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:51:30.231629 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:51:30.231687 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:51:30.231688 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:51:30.232521 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:51:30.232522 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:51:30.233096 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:51:30.233098 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:51:30.233590 (+ 492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:51:30.233591 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:51:30.233656 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:30.234106 (+ 450us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:51:30.234111 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:51:30.235744 (+ 1633us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:51:30.237653 (+ 1909us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:51:30.237654 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:51:30.237654 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:51:30.237938 (+ 284us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:51:30.238147 (+ 209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:51:30.238148 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:51:30.238148 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:51:30.238224 (+ 76us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:51:30.238497 (+ 273us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:51:30.238498 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:51:30.238550 (+ 52us) client_negotiation.cc:769] Sending connection context
0806 03:51:30.238632 (+ 82us) client_negotiation.cc:240] Negotiation successful
0806 03:51:30.238744 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48}
I20260806 03:51:30.239929 11787 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.253.252" port: 35693 } has no permanent_uuid. Determining permanent_uuid...
I20260806 03:51:30.240319 11789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:30.240171 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.252:35693 (local address 127.0.0.1:52250)
0806 03:51:30.240227 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0806 03:51:30.240284 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.253.252:35693: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260806 03:51:30.240473 11721 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.11.253.252:35693: connect: Connection refused (error 111)
I20260806 03:51:30.240952 11789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:30.240840 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.252:35693 (local address 127.0.0.1:52264)
0806 03:51:30.240889 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0806 03:51:30.240923 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.253.252:35693: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260806 03:51:30.241074 11787 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.253.252:35693: Network error: Client connection negotiation failed: client connection to 127.11.253.252:35693: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.253.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.252.keytab.
Entry for principal kudu/127.11.253.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.252.keytab.
I20260806 03:51:30.252596 12279 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskToVWRe/build/release/bin/kudu
/tmp/dist-test-taskToVWRe/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785988004637995-12279-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785988004637995-12279-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785988004637995-12279-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785988004637995-12279-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.253.252:35693
--webserver_interface=127.11.253.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.252.keytab
--principal=kudu/127.11.253.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.253.212:41027
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.253.254:35715,127.11.253.253:39559,127.11.253.252:35693
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785988004637995-12279-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785988004637995-12279-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785988004637995-12279-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 03:51:30.277084 11683 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.253.253" port: 39559 } attempt: 3
Aug 06 03:51:30 dist-test-slave-6ntk krb5kdc[11570](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988290, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.253.254@KRBTEST.COM for kudu/127.11.253.253@KRBTEST.COM
I20260806 03:51:30.284972 11791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:30.277632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52058 (local address 127.11.253.253:39559)
0806 03:51:30.277702 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0806 03:51:30.277704 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:51:30.277713 (+ 9us) server_negotiation.cc:407] Connection header received
0806 03:51:30.277741 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:51:30.277742 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:51:30.277760 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:51:30.277827 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:51:30.278201 (+ 374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:30.278713 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:30.279573 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:30.279943 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:30.280009 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:30.282229 (+ 2220us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:51:30.282231 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:51:30.282231 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:51:30.282239 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:51:30.283751 (+ 1512us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:51:30.284137 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:51:30.284138 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:51:30.284139 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:51:30.284202 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:51:30.284418 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:51:30.284419 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:51:30.284420 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:51:30.284543 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:51:30.284618 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0806 03:51:30.284788 (+ 170us) server_negotiation.cc:299] Negotiation successful
0806 03:51:30.284867 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260806 03:51:30.285010 11687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:30.277528 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.253:39559 (local address 127.0.0.1:52058)
0806 03:51:30.277593 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0806 03:51:30.277603 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 03:51:30.277645 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:51:30.277835 (+ 190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:51:30.277836 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:51:30.278046 (+ 210us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:51:30.278114 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:51:30.278116 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:51:30.278804 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:51:30.278805 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:51:30.279445 (+ 640us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:51:30.279448 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:51:30.280030 (+ 582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:51:30.280031 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:51:30.280151 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:30.280611 (+ 460us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:51:30.280616 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:51:30.282136 (+ 1520us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:51:30.283859 (+ 1723us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:51:30.283860 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:51:30.283861 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:51:30.284051 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:51:30.284278 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:51:30.284279 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:51:30.284279 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:51:30.284348 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:51:30.284621 (+ 273us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:51:30.284622 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:51:30.284671 (+ 49us) client_negotiation.cc:769] Sending connection context
0806 03:51:30.284751 (+ 80us) client_negotiation.cc:240] Negotiation successful
0806 03:51:30.284865 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260806 03:51:30.285648 11683 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.253.252" port: 35693 } has no permanent_uuid. Determining permanent_uuid...
I20260806 03:51:30.286103 11687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:30.285991 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.252:35693 (local address 127.0.0.1:52268)
0806 03:51:30.286045 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0806 03:51:30.286079 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.253.252:35693: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260806 03:51:30.286458 11687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:30.286374 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.252:35693 (local address 127.0.0.1:52276)
0806 03:51:30.286419 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0806 03:51:30.286443 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.253.252:35693: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260806 03:51:30.286567 11683 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.253.252:35693: Network error: Client connection negotiation failed: client connection to 127.11.253.252:35693: connect: Connection refused (error 111)
I20260806 03:51:30.290306 11787 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.253.252" port: 35693 } attempt: 1
I20260806 03:51:30.290735 11789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:30.290606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.252:35693 (local address 127.0.0.1:52280)
0806 03:51:30.290667 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0806 03:51:30.290704 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.253.252:35693: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260806 03:51:30.291098 11789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:30.291004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.252:35693 (local address 127.0.0.1:52292)
0806 03:51:30.291051 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0806 03:51:30.291077 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.253.252:35693: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260806 03:51:30.291242 11787 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.253.252:35693: Network error: Client connection negotiation failed: client connection to 127.11.253.252:35693: connect: Connection refused (error 111)
I20260806 03:51:30.317721 11683 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.253.252" port: 35693 } attempt: 1
I20260806 03:51:30.318280 11687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:30.318099 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.252:35693 (local address 127.0.0.1:52298)
0806 03:51:30.318180 (+ 81us) negotiation.cc:107] Waiting for socket to connect
0806 03:51:30.318232 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.253.252:35693: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260806 03:51:30.318706 11687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:30.318603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.252:35693 (local address 127.0.0.1:52314)
0806 03:51:30.318651 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0806 03:51:30.318681 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.253.252:35693: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260806 03:51:30.318840 11683 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.253.252:35693: Network error: Client connection negotiation failed: client connection to 127.11.253.252:35693: connect: Connection refused (error 111)
W20260806 03:51:30.339545 11796 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 03:51:30.339768 11796 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:51:30.339785 11796 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:51:30.339833 11796 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:51:30.341014 11796 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 03:51:30.341058 11796 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:51:30.341073 11796 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:51:30.341123 11796 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 03:51:30.341142 11796 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 03:51:30.342737 11796 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.253.212:41027
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785988004637995-12279-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785988004637995-12279-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.11.253.254:35715,127.11.253.253:39559,127.11.253.252:35693
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785988004637995-12279-0/krb5kdc/kudu_127.11.253.252.keytab
--principal=kudu/127.11.253.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.253.252:35693
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785988004637995-12279-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.11.253.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11796
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785988004637995-12279-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:51:30.343012 11796 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:51:30.343257 11796 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:51:30.348344 11805 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:51:30.348349 11804 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:51:30.348465 11796 server_base.cc:1034] running on GCE node
W20260806 03:51:30.348357 11807 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:51:30.348896 11796 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:51:30.349152 11796 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:51:30.350301 11796 hybrid_clock.cc:648] HybridClock initialized: now 1785988290350281 us; error 41 us; skew 500 ppm
Aug 06 03:51:30 dist-test-slave-6ntk krb5kdc[11570](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988290, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.253.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:51:30.352869 11796 init.cc:377] Logged in from keytab as kudu/127.11.253.252@KRBTEST.COM (short username kudu)
I20260806 03:51:30.353379 11796 webserver.cc:460] Webserver started at http://127.11.253.252:46611/ using document root <none> and password file <none>
I20260806 03:51:30.353616 11796 fs_manager.cc:359] Metadata directory not provided
I20260806 03:51:30.353667 11796 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:51:30.353825 11796 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:51:30.354984 11796 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785988004637995-12279-0/minicluster-data/master-2/data/instance:
uuid: "a3235ea9fa554affbd4f3758620ad411"
format_stamp: "Formatted at 2026-08-06 03:51:30 on dist-test-slave-6ntk"
server_key: "fff1a7af8a7cab7560a5d71ae76d0747"
server_key_iv: "95976248976215192ed725a72ba513ed"
server_key_version: "encryptionkey@0"
I20260806 03:51:30.355360 11796 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785988004637995-12279-0/minicluster-data/master-2/wal/instance:
uuid: "a3235ea9fa554affbd4f3758620ad411"
format_stamp: "Formatted at 2026-08-06 03:51:30 on dist-test-slave-6ntk"
server_key: "fff1a7af8a7cab7560a5d71ae76d0747"
server_key_iv: "95976248976215192ed725a72ba513ed"
server_key_version: "encryptionkey@0"
I20260806 03:51:30.356725 11796 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 03:51:30.357448 11817 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:51:30.357635 11796 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 03:51:30.357743 11796 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785988004637995-12279-0/minicluster-data/master-2/data,/tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785988004637995-12279-0/minicluster-data/master-2/wal
uuid: "a3235ea9fa554affbd4f3758620ad411"
format_stamp: "Formatted at 2026-08-06 03:51:30 on dist-test-slave-6ntk"
server_key: "fff1a7af8a7cab7560a5d71ae76d0747"
server_key_iv: "95976248976215192ed725a72ba513ed"
server_key_version: "encryptionkey@0"
I20260806 03:51:30.357826 11796 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785988004637995-12279-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785988004637995-12279-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785988004637995-12279-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:51:30.359403 11787 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.253.252" port: 35693 } attempt: 2
I20260806 03:51:30.359778 11789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:30.359656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.252:35693 (local address 127.0.0.1:52324)
0806 03:51:30.359714 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0806 03:51:30.359750 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.253.252:35693: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260806 03:51:30.360112 11789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:30.360018 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.252:35693 (local address 127.0.0.1:52326)
0806 03:51:30.360062 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0806 03:51:30.360090 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.253.252:35693: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260806 03:51:30.360193 11787 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.253.252:35693: Network error: Client connection negotiation failed: client connection to 127.11.253.252:35693: connect: Connection refused (error 111)
I20260806 03:51:30.364970 11796 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:51:30.366873 11796 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:51:30.367031 11796 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:51:30.371196 11796 rpc_server.cc:307] RPC server started. Bound to: 127.11.253.252:35693
I20260806 03:51:30.371225 11892 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.253.252:35693 every 8 connection(s)
I20260806 03:51:30.371575 11796 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785988004637995-12279-0/minicluster-data/master-2/data/info.pb
I20260806 03:51:30.372435 11893 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:51:30.374003 11893 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.253.254" port: 35715 } has no permanent_uuid. Determining permanent_uuid...
I20260806 03:51:30.378062 12279 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskToVWRe/build/release/bin/kudu as pid 11796
I20260806 03:51:30.378159 12279 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskToVWRe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785988004637995-12279-0/minicluster-data/master-2/wal/instance
I20260806 03:51:30.378379 12279 external_mini_cluster.cc:1428] Setting key d5db8d85a056815f4a8ffd30cd472d6d
Aug 06 03:51:30 dist-test-slave-6ntk krb5kdc[11570](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988290, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.253.252@KRBTEST.COM for kudu/127.11.253.254@KRBTEST.COM
I20260806 03:51:30.384375 11688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:30.374707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43066 (local address 127.11.253.254:35715)
0806 03:51:30.374789 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0806 03:51:30.374792 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 03:51:30.375036 (+ 244us) server_negotiation.cc:407] Connection header received
0806 03:51:30.375604 (+ 568us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:51:30.375605 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:51:30.375615 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:51:30.375665 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:51:30.376375 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:30.376844 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:30.378089 (+ 1245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:30.378399 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:30.378506 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:30.380970 (+ 2464us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:51:30.380972 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:51:30.380972 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:51:30.380980 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:51:30.382782 (+ 1802us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:51:30.383232 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:51:30.383233 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:51:30.383233 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:51:30.383303 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:51:30.383579 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:51:30.383580 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:51:30.383580 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:51:30.383743 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:51:30.383811 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0806 03:51:30.384231 (+ 420us) server_negotiation.cc:299] Negotiation successful
0806 03:51:30.384308 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
Aug 06 03:51:30 dist-test-slave-6ntk krb5kdc[11570](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988289, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.253.252@KRBTEST.COM
I20260806 03:51:30.384608 11896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:30.374662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.254:35715 (local address 127.0.0.1:43066)
0806 03:51:30.374894 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0806 03:51:30.374941 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0806 03:51:30.375503 (+ 562us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:51:30.375735 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:51:30.375738 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:51:30.376015 (+ 277us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:51:30.376281 (+ 266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:51:30.376286 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:51:30.376960 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:51:30.376961 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:51:30.377972 (+ 1011us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:51:30.377975 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:51:30.378517 (+ 542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:51:30.378519 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:51:30.378667 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:30.379297 (+ 630us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:51:30.379306 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:51:30.380877 (+ 1571us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:51:30.382873 (+ 1996us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:51:30.382881 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:51:30.382882 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:51:30.383141 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:51:30.383383 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:51:30.383384 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:51:30.383384 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:51:30.383501 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:51:30.383851 (+ 350us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:51:30.383861 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:51:30.384067 (+ 206us) client_negotiation.cc:769] Sending connection context
0806 03:51:30.384204 (+ 137us) client_negotiation.cc:240] Negotiation successful
0806 03:51:30.384364 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":118,"threads_started":1}
I20260806 03:51:30.385313 11893 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.253.253" port: 39559 } has no permanent_uuid. Determining permanent_uuid...
I20260806 03:51:30.388089 11898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:30.378758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52334 (local address 127.11.253.252:35693)
0806 03:51:30.378959 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0806 03:51:30.378963 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 03:51:30.379124 (+ 161us) server_negotiation.cc:407] Connection header received
0806 03:51:30.379184 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:51:30.379188 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:51:30.379210 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:51:30.379304 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:51:30.379911 (+ 607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:30.380615 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:30.381587 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:30.382068 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:30.382134 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:30.385033 (+ 2899us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:51:30.385041 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:51:30.385041 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:51:30.385049 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:51:30.386795 (+ 1746us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:51:30.387181 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:51:30.387184 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:51:30.387184 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:51:30.387238 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:51:30.387466 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:51:30.387467 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:51:30.387467 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:51:30.387616 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:51:30.387807 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0806 03:51:30.387885 (+ 78us) server_negotiation.cc:299] Negotiation successful
0806 03:51:30.387966 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":176,"thread_start_us":111,"threads_started":1}
Aug 06 03:51:30 dist-test-slave-6ntk krb5kdc[11570](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988290, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.253.252@KRBTEST.COM for kudu/127.11.253.253@KRBTEST.COM
I20260806 03:51:30.392956 11896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:30.385608 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.253:39559 (local address 127.0.0.1:52068)
0806 03:51:30.385662 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0806 03:51:30.385672 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 03:51:30.385712 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:51:30.385840 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:51:30.385840 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:51:30.386039 (+ 199us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:51:30.386098 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:51:30.386100 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:51:30.386727 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:51:30.386728 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:51:30.387538 (+ 810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:51:30.387540 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:51:30.387971 (+ 431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:51:30.387973 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:51:30.388048 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:30.388694 (+ 646us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:51:30.388699 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:51:30.390172 (+ 1473us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:51:30.391927 (+ 1755us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:51:30.391929 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:51:30.391929 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:51:30.392104 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:51:30.392309 (+ 205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:51:30.392310 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:51:30.392310 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:51:30.392369 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:51:30.392628 (+ 259us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:51:30.392629 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:51:30.392673 (+ 44us) client_negotiation.cc:769] Sending connection context
0806 03:51:30.392755 (+ 82us) client_negotiation.cc:240] Negotiation successful
0806 03:51:30.392853 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260806 03:51:30.392956 11791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:30.385679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52068 (local address 127.11.253.253:39559)
0806 03:51:30.385743 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0806 03:51:30.385746 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 03:51:30.385754 (+ 8us) server_negotiation.cc:407] Connection header received
0806 03:51:30.385779 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:51:30.385780 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:51:30.385789 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:51:30.385839 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:51:30.386185 (+ 346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:30.386647 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:30.387623 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:30.387857 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:30.387935 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:30.390267 (+ 2332us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:51:30.390268 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:51:30.390269 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:51:30.390276 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:51:30.391821 (+ 1545us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:51:30.392177 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:51:30.392178 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:51:30.392178 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:51:30.392241 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:51:30.392443 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:51:30.392444 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:51:30.392444 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:51:30.392562 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:51:30.392636 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0806 03:51:30.392784 (+ 148us) server_negotiation.cc:299] Negotiation successful
0806 03:51:30.392857 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
Aug 06 03:51:30 dist-test-slave-6ntk krb5kdc[11570](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988289, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.253.254@KRBTEST.COM
I20260806 03:51:30.393741 11893 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.253.252" port: 35693 } has no permanent_uuid. Determining permanent_uuid...
I20260806 03:51:30.394131 11683 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.253.252" port: 35693 } attempt: 2
I20260806 03:51:30.397143 11688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:30.389053 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43068 (local address 127.11.253.254:35715)
0806 03:51:30.389097 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0806 03:51:30.389099 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:51:30.389108 (+ 9us) server_negotiation.cc:407] Connection header received
0806 03:51:30.389131 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:51:30.389132 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:51:30.389143 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:51:30.389203 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:51:30.389810 (+ 607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:30.390350 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:30.391015 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:30.391207 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:30.391284 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:30.394028 (+ 2744us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:51:30.394030 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:51:30.394030 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:51:30.394038 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:51:30.395779 (+ 1741us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:51:30.396126 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:51:30.396127 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:51:30.396127 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:51:30.396189 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:51:30.396425 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:51:30.396426 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:51:30.396426 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:51:30.396549 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:51:30.396718 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0806 03:51:30.396930 (+ 212us) server_negotiation.cc:299] Negotiation successful
0806 03:51:30.397041 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
Aug 06 03:51:30 dist-test-slave-6ntk krb5kdc[11570](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988290, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.253.252@KRBTEST.COM for kudu/127.11.253.252@KRBTEST.COM
Aug 06 03:51:30 dist-test-slave-6ntk krb5kdc[11570](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988290, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.253.254@KRBTEST.COM for kudu/127.11.253.252@KRBTEST.COM
I20260806 03:51:30.401387 11896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:30.394084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.252:35693 (local address 127.0.0.1:52336)
0806 03:51:30.394192 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0806 03:51:30.394204 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 03:51:30.394311 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:51:30.394445 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:51:30.394445 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:51:30.394673 (+ 228us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:51:30.394713 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:51:30.394715 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:51:30.395385 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:51:30.395386 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:51:30.395839 (+ 453us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:51:30.395841 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:51:30.396281 (+ 440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:51:30.396282 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:51:30.396330 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:30.396732 (+ 402us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:51:30.396736 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:51:30.398173 (+ 1437us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:51:30.400461 (+ 2288us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:51:30.400462 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:51:30.400463 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:51:30.400625 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:51:30.400831 (+ 206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:51:30.400832 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:51:30.400833 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:51:30.400916 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:51:30.401182 (+ 266us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:51:30.401184 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:51:30.401217 (+ 33us) client_negotiation.cc:769] Sending connection context
0806 03:51:30.401264 (+ 47us) client_negotiation.cc:240] Negotiation successful
0806 03:51:30.401329 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":86}
I20260806 03:51:30.401713 11898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:30.394280 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52336 (local address 127.11.253.252:35693)
0806 03:51:30.394322 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0806 03:51:30.394324 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:51:30.394332 (+ 8us) server_negotiation.cc:407] Connection header received
0806 03:51:30.394353 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:51:30.394353 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:51:30.394366 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:51:30.394429 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:51:30.394783 (+ 354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:30.395295 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:30.395937 (+ 642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:30.396225 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:30.396285 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:30.398239 (+ 1954us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:51:30.398240 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:51:30.398241 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:51:30.398247 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:51:30.400356 (+ 2109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:51:30.400697 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:51:30.400698 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:51:30.400698 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:51:30.400747 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:51:30.400987 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:51:30.400988 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:51:30.400988 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:51:30.401108 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:51:30.401517 (+ 409us) server_negotiation.cc:1035] Waiting for connection context
0806 03:51:30.401562 (+ 45us) server_negotiation.cc:299] Negotiation successful
0806 03:51:30.401641 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
Aug 06 03:51:30 dist-test-slave-6ntk krb5kdc[11570](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988289, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.253.253@KRBTEST.COM
I20260806 03:51:30.403478 11893 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a3235ea9fa554affbd4f3758620ad411: Bootstrap starting.
I20260806 03:51:30.403985 11687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:30.394618 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.252:35693 (local address 127.0.0.1:52342)
0806 03:51:30.394678 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0806 03:51:30.394688 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 03:51:30.394728 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:51:30.395966 (+ 1238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:51:30.395967 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:51:30.396121 (+ 154us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:51:30.396161 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:51:30.396162 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:51:30.397085 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:51:30.397086 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:51:30.397766 (+ 680us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:51:30.397768 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:51:30.398331 (+ 563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:51:30.398333 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:51:30.398400 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:30.398928 (+ 528us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:51:30.398933 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:51:30.400150 (+ 1217us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:51:30.402755 (+ 2605us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:51:30.402756 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:51:30.402757 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:51:30.402984 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:51:30.403228 (+ 244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:51:30.403229 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:51:30.403229 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:51:30.403298 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:51:30.403639 (+ 341us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:51:30.403640 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:51:30.403689 (+ 49us) client_negotiation.cc:769] Sending connection context
0806 03:51:30.403763 (+ 74us) client_negotiation.cc:240] Negotiation successful
0806 03:51:30.403857 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43}
I20260806 03:51:30.404258 11899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:30.394798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52342 (local address 127.11.253.252:35693)
0806 03:51:30.395225 (+ 427us) server_negotiation.cc:206] Beginning negotiation
0806 03:51:30.395227 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:51:30.395236 (+ 9us) server_negotiation.cc:407] Connection header received
0806 03:51:30.395256 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:51:30.395257 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:51:30.395267 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:51:30.395377 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:51:30.396246 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:30.396987 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:30.397912 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:30.398266 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:30.398334 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:30.400317 (+ 1983us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:51:30.400318 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:51:30.400318 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:51:30.400325 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:51:30.402666 (+ 2341us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:51:30.403074 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:51:30.403075 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:51:30.403075 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:51:30.403138 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:51:30.403358 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:51:30.403359 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:51:30.403359 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:51:30.403516 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:51:30.403642 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0806 03:51:30.403802 (+ 160us) server_negotiation.cc:299] Negotiation successful
0806 03:51:30.403894 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":404,"thread_start_us":94,"threads_started":1}
I20260806 03:51:30.404383 11893 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a3235ea9fa554affbd4f3758620ad411: Neither blocks nor log segments found. Creating new log.
I20260806 03:51:30.404791 11893 log.cc:824] T 00000000000000000000000000000000 P a3235ea9fa554affbd4f3758620ad411: Log is configured to *not* fsync() on all Append() calls
I20260806 03:51:30.405731 11893 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a3235ea9fa554affbd4f3758620ad411: No bootstrap required, opened a new log
I20260806 03:51:30.406584 11791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:30.398340 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52078 (local address 127.11.253.253:39559)
0806 03:51:30.398375 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0806 03:51:30.398377 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:51:30.398478 (+ 101us) server_negotiation.cc:407] Connection header received
0806 03:51:30.398560 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:51:30.398561 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:51:30.398569 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:51:30.398665 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:51:30.399277 (+ 612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:30.399783 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:30.400773 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:30.401088 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:30.401161 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:30.403748 (+ 2587us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:51:30.403749 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:51:30.403750 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:51:30.403757 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:51:30.405348 (+ 1591us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:51:30.405737 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:51:30.405739 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:51:30.405739 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:51:30.405799 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:51:30.406066 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:51:30.406067 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:51:30.406067 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:51:30.406193 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:51:30.406374 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0806 03:51:30.406413 (+ 39us) server_negotiation.cc:299] Negotiation successful
0806 03:51:30.406490 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260806 03:51:30.407047 11683 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dbdeb40a549e4c27ba6f21fa8f59c37c: Bootstrap starting.
I20260806 03:51:30.407747 11683 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dbdeb40a549e4c27ba6f21fa8f59c37c: Neither blocks nor log segments found. Creating new log.
I20260806 03:51:30.407833 11893 raft_consensus.cc:348] T 00000000000000000000000000000000 P a3235ea9fa554affbd4f3758620ad411 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbdeb40a549e4c27ba6f21fa8f59c37c" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 35715 } } peers { permanent_uuid: "9f42bb2328ab469ea6b81aa84464da5e" member_type: VOTER last_known_addr { host: "127.11.253.253" port: 39559 } } peers { permanent_uuid: "a3235ea9fa554affbd4f3758620ad411" member_type: VOTER last_known_addr { host: "127.11.253.252" port: 35693 } }
I20260806 03:51:30.407948 11893 raft_consensus.cc:729] T 00000000000000000000000000000000 P a3235ea9fa554affbd4f3758620ad411 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3235ea9fa554affbd4f3758620ad411, State: Initialized, Role: FOLLOWER
I20260806 03:51:30.408002 11893 consensus_queue.cc:260] T 00000000000000000000000000000000 P a3235ea9fa554affbd4f3758620ad411 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbdeb40a549e4c27ba6f21fa8f59c37c" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 35715 } } peers { permanent_uuid: "9f42bb2328ab469ea6b81aa84464da5e" member_type: VOTER last_known_addr { host: "127.11.253.253" port: 39559 } } peers { permanent_uuid: "a3235ea9fa554affbd4f3758620ad411" member_type: VOTER last_known_addr { host: "127.11.253.252" port: 35693 } }
I20260806 03:51:30.408077 11683 log.cc:824] T 00000000000000000000000000000000 P dbdeb40a549e4c27ba6f21fa8f59c37c: Log is configured to *not* fsync() on all Append() calls
I20260806 03:51:30.408140 12279 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 03:51:30.408257 11903 sys_catalog.cc:455] T 00000000000000000000000000000000 P a3235ea9fa554affbd4f3758620ad411 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbdeb40a549e4c27ba6f21fa8f59c37c" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 35715 } } peers { permanent_uuid: "9f42bb2328ab469ea6b81aa84464da5e" member_type: VOTER last_known_addr { host: "127.11.253.253" port: 39559 } } peers { permanent_uuid: "a3235ea9fa554affbd4f3758620ad411" member_type: VOTER last_known_addr { host: "127.11.253.252" port: 35693 } } }
I20260806 03:51:30.408326 11893 sys_catalog.cc:564] T 00000000000000000000000000000000 P a3235ea9fa554affbd4f3758620ad411 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 03:51:30.408344 11903 sys_catalog.cc:458] T 00000000000000000000000000000000 P a3235ea9fa554affbd4f3758620ad411 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 03:51:30.409530 11683 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dbdeb40a549e4c27ba6f21fa8f59c37c: No bootstrap required, opened a new log
I20260806 03:51:30.409051 11642 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "dbdeb40a549e4c27ba6f21fa8f59c37c"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43068
W20260806 03:51:30.410219 11918 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a3235ea9fa554affbd4f3758620ad411: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 03:51:30.410266 11918 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 03:51:30.411228 11683 raft_consensus.cc:348] T 00000000000000000000000000000000 P dbdeb40a549e4c27ba6f21fa8f59c37c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbdeb40a549e4c27ba6f21fa8f59c37c" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 35715 } } peers { permanent_uuid: "9f42bb2328ab469ea6b81aa84464da5e" member_type: VOTER last_known_addr { host: "127.11.253.253" port: 39559 } } peers { permanent_uuid: "a3235ea9fa554affbd4f3758620ad411" member_type: VOTER last_known_addr { host: "127.11.253.252" port: 35693 } }
I20260806 03:51:30.411361 11683 raft_consensus.cc:729] T 00000000000000000000000000000000 P dbdeb40a549e4c27ba6f21fa8f59c37c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dbdeb40a549e4c27ba6f21fa8f59c37c, State: Initialized, Role: FOLLOWER
I20260806 03:51:30.411440 11683 consensus_queue.cc:260] T 00000000000000000000000000000000 P dbdeb40a549e4c27ba6f21fa8f59c37c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbdeb40a549e4c27ba6f21fa8f59c37c" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 35715 } } peers { permanent_uuid: "9f42bb2328ab469ea6b81aa84464da5e" member_type: VOTER last_known_addr { host: "127.11.253.253" port: 39559 } } peers { permanent_uuid: "a3235ea9fa554affbd4f3758620ad411" member_type: VOTER last_known_addr { host: "127.11.253.252" port: 35693 } }
I20260806 03:51:30.411715 11683 sys_catalog.cc:564] T 00000000000000000000000000000000 P dbdeb40a549e4c27ba6f21fa8f59c37c [sys.catalog]: configured and running, proceeding with master startup.
I20260806 03:51:30.411684 11920 sys_catalog.cc:455] T 00000000000000000000000000000000 P dbdeb40a549e4c27ba6f21fa8f59c37c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbdeb40a549e4c27ba6f21fa8f59c37c" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 35715 } } peers { permanent_uuid: "9f42bb2328ab469ea6b81aa84464da5e" member_type: VOTER last_known_addr { host: "127.11.253.253" port: 39559 } } peers { permanent_uuid: "a3235ea9fa554affbd4f3758620ad411" member_type: VOTER last_known_addr { host: "127.11.253.252" port: 35693 } } }
I20260806 03:51:30.411752 11642 raft_consensus.cc:482] T 00000000000000000000000000000000 P dbdeb40a549e4c27ba6f21fa8f59c37c [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260806 03:51:30.411759 11920 sys_catalog.cc:458] T 00000000000000000000000000000000 P dbdeb40a549e4c27ba6f21fa8f59c37c [sys.catalog]: This master's current role is: FOLLOWER
I20260806 03:51:30.411827 11642 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dbdeb40a549e4c27ba6f21fa8f59c37c [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:51:30.412599 11642 raft_consensus.cc:504] T 00000000000000000000000000000000 P dbdeb40a549e4c27ba6f21fa8f59c37c [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbdeb40a549e4c27ba6f21fa8f59c37c" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 35715 } } peers { permanent_uuid: "9f42bb2328ab469ea6b81aa84464da5e" member_type: VOTER last_known_addr { host: "127.11.253.253" port: 39559 } } peers { permanent_uuid: "a3235ea9fa554affbd4f3758620ad411" member_type: VOTER last_known_addr { host: "127.11.253.252" port: 35693 } }
I20260806 03:51:30.412834 11642 leader_election.cc:288] T 00000000000000000000000000000000 P dbdeb40a549e4c27ba6f21fa8f59c37c [CANDIDATE]: Term 1 election: Requested vote from peers 9f42bb2328ab469ea6b81aa84464da5e (127.11.253.253:39559), a3235ea9fa554affbd4f3758620ad411 (127.11.253.252:35693)
I20260806 03:51:30.410815 11796 master_runner.cc:186] Error getting master registration for 127.11.253.254:35715: OK, instance_id { permanent_uuid: "dbdeb40a549e4c27ba6f21fa8f59c37c" instance_seqno: 1785988290048410 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 03:51:30.413650 11796 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 03:51:30.414346 11853 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "dbdeb40a549e4c27ba6f21fa8f59c37c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "a3235ea9fa554affbd4f3758620ad411"
I20260806 03:51:30.414443 11853 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a3235ea9fa554affbd4f3758620ad411 [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:51:30.415077 11853 raft_consensus.cc:2454] T 00000000000000000000000000000000 P a3235ea9fa554affbd4f3758620ad411 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbdeb40a549e4c27ba6f21fa8f59c37c in term 1.
I20260806 03:51:30.415310 11615 leader_election.cc:302] T 00000000000000000000000000000000 P dbdeb40a549e4c27ba6f21fa8f59c37c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a3235ea9fa554affbd4f3758620ad411, dbdeb40a549e4c27ba6f21fa8f59c37c; no voters:
I20260806 03:51:30.415431 11920 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dbdeb40a549e4c27ba6f21fa8f59c37c [term 1 FOLLOWER]: Leader election won for term 1
I20260806 03:51:30.415575 11920 raft_consensus.cc:686] T 00000000000000000000000000000000 P dbdeb40a549e4c27ba6f21fa8f59c37c [term 1 LEADER]: Becoming Leader. State: Replica: dbdeb40a549e4c27ba6f21fa8f59c37c, State: Running, Role: LEADER
I20260806 03:51:30.415652 11920 consensus_queue.cc:237] T 00000000000000000000000000000000 P dbdeb40a549e4c27ba6f21fa8f59c37c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbdeb40a549e4c27ba6f21fa8f59c37c" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 35715 } } peers { permanent_uuid: "9f42bb2328ab469ea6b81aa84464da5e" member_type: VOTER last_known_addr { host: "127.11.253.253" port: 39559 } } peers { permanent_uuid: "a3235ea9fa554affbd4f3758620ad411" member_type: VOTER last_known_addr { host: "127.11.253.252" port: 35693 } }
I20260806 03:51:30.416066 11920 sys_catalog.cc:455] T 00000000000000000000000000000000 P dbdeb40a549e4c27ba6f21fa8f59c37c [sys.catalog]: SysCatalogTable state changed. Reason: New leader dbdeb40a549e4c27ba6f21fa8f59c37c. Latest consensus state: current_term: 1 leader_uuid: "dbdeb40a549e4c27ba6f21fa8f59c37c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbdeb40a549e4c27ba6f21fa8f59c37c" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 35715 } } peers { permanent_uuid: "9f42bb2328ab469ea6b81aa84464da5e" member_type: VOTER last_known_addr { host: "127.11.253.253" port: 39559 } } peers { permanent_uuid: "a3235ea9fa554affbd4f3758620ad411" member_type: VOTER last_known_addr { host: "127.11.253.252" port: 35693 } } }
I20260806 03:51:30.416144 11920 sys_catalog.cc:458] T 00000000000000000000000000000000 P dbdeb40a549e4c27ba6f21fa8f59c37c [sys.catalog]: This master's current role is: LEADER
I20260806 03:51:30.417050 11949 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 03:51:30.417855 11949 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 03:51:30.416566 11592 master_runner.cc:186] Error getting master registration for 127.11.253.253:39559: OK, instance_id { permanent_uuid: "9f42bb2328ab469ea6b81aa84464da5e" instance_seqno: 1785988290215417 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 03:51:30.419380 11592 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 03:51:30.424165 11791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:30.414025 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52092 (local address 127.11.253.253:39559)
0806 03:51:30.414066 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0806 03:51:30.414069 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 03:51:30.416352 (+ 2283us) server_negotiation.cc:407] Connection header received
0806 03:51:30.416459 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:51:30.416460 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:51:30.416471 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:51:30.416501 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:51:30.417360 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:30.418100 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:30.419128 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:30.419430 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:30.419496 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:30.421098 (+ 1602us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:51:30.421099 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:51:30.421100 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:51:30.421106 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:51:30.422855 (+ 1749us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:51:30.423247 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:51:30.423248 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:51:30.423249 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:51:30.423302 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:51:30.423505 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:51:30.423505 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:51:30.423506 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:51:30.423633 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:51:30.423729 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0806 03:51:30.423968 (+ 239us) server_negotiation.cc:299] Negotiation successful
0806 03:51:30.424056 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260806 03:51:30.426359 11853 raft_consensus.cc:1261] T 00000000000000000000000000000000 P a3235ea9fa554affbd4f3758620ad411 [term 1 FOLLOWER]: Refusing update from remote peer dbdeb40a549e4c27ba6f21fa8f59c37c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 03:51:30.426789 11920 consensus_queue.cc:1035] T 00000000000000000000000000000000 P dbdeb40a549e4c27ba6f21fa8f59c37c [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3235ea9fa554affbd4f3758620ad411" member_type: VOTER last_known_addr { host: "127.11.253.252" port: 35693 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 03:51:30.428160 11903 sys_catalog.cc:455] T 00000000000000000000000000000000 P a3235ea9fa554affbd4f3758620ad411 [sys.catalog]: SysCatalogTable state changed. Reason: New leader dbdeb40a549e4c27ba6f21fa8f59c37c. Latest consensus state: current_term: 1 leader_uuid: "dbdeb40a549e4c27ba6f21fa8f59c37c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbdeb40a549e4c27ba6f21fa8f59c37c" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 35715 } } peers { permanent_uuid: "9f42bb2328ab469ea6b81aa84464da5e" member_type: VOTER last_known_addr { host: "127.11.253.253" port: 39559 } } peers { permanent_uuid: "a3235ea9fa554affbd4f3758620ad411" member_type: VOTER last_known_addr { host: "127.11.253.252" port: 35693 } } }
I20260806 03:51:30.428247 11903 sys_catalog.cc:458] T 00000000000000000000000000000000 P a3235ea9fa554affbd4f3758620ad411 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 03:51:30.428807 11688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:30.414109 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43080 (local address 127.11.253.254:35715)
0806 03:51:30.414155 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0806 03:51:30.414157 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:51:30.419252 (+ 5095us) server_negotiation.cc:407] Connection header received
0806 03:51:30.419547 (+ 295us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:51:30.419548 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:51:30.419558 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:51:30.419588 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:51:30.420685 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:30.421763 (+ 1078us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:30.423080 (+ 1317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:30.423396 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:30.423436 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:30.425242 (+ 1806us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:51:30.425243 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:51:30.425243 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:51:30.425251 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:51:30.426976 (+ 1725us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:51:30.427481 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:51:30.427483 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:51:30.427483 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:51:30.427544 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:51:30.428042 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:51:30.428043 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:51:30.428044 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:51:30.428190 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:51:30.428291 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0806 03:51:30.428635 (+ 344us) server_negotiation.cc:299] Negotiation successful
0806 03:51:30.428710 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260806 03:51:30.430078 11920 sys_catalog.cc:455] T 00000000000000000000000000000000 P dbdeb40a549e4c27ba6f21fa8f59c37c [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "dbdeb40a549e4c27ba6f21fa8f59c37c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbdeb40a549e4c27ba6f21fa8f59c37c" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 35715 } } peers { permanent_uuid: "9f42bb2328ab469ea6b81aa84464da5e" member_type: VOTER last_known_addr { host: "127.11.253.253" port: 39559 } } peers { permanent_uuid: "a3235ea9fa554affbd4f3758620ad411" member_type: VOTER last_known_addr { host: "127.11.253.252" port: 35693 } } }
I20260806 03:51:30.430172 11920 sys_catalog.cc:458] T 00000000000000000000000000000000 P dbdeb40a549e4c27ba6f21fa8f59c37c [sys.catalog]: This master's current role is: LEADER
I20260806 03:51:30.430171 11903 sys_catalog.cc:455] T 00000000000000000000000000000000 P a3235ea9fa554affbd4f3758620ad411 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "dbdeb40a549e4c27ba6f21fa8f59c37c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbdeb40a549e4c27ba6f21fa8f59c37c" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 35715 } } peers { permanent_uuid: "9f42bb2328ab469ea6b81aa84464da5e" member_type: VOTER last_known_addr { host: "127.11.253.253" port: 39559 } } peers { permanent_uuid: "a3235ea9fa554affbd4f3758620ad411" member_type: VOTER last_known_addr { host: "127.11.253.252" port: 35693 } } }
I20260806 03:51:30.430251 11903 sys_catalog.cc:458] T 00000000000000000000000000000000 P a3235ea9fa554affbd4f3758620ad411 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 03:51:30.430467 11953 mvcc.cc:205] Tried to move back new op lower bound from 7315408037581537280 to 7315408037543088128. Current Snapshot: MvccSnapshot[applied={T|T < 7315408037581537280}]
I20260806 03:51:30.430806 11949 catalog_manager.cc:1347] Generated new cluster ID: 42700d7593994432bca26b908debdcfc
I20260806 03:51:30.430851 11949 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 03:51:30.431000 11952 mvcc.cc:205] Tried to move back new op lower bound from 7315408037581537280 to 7315408037543088128. Current Snapshot: MvccSnapshot[applied={T|T < 7315408037581537280 or (T in {7315408037581537280})}]
I20260806 03:51:30.432185 11899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:30.415836 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52356 (local address 127.11.253.252:35693)
0806 03:51:30.415883 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0806 03:51:30.415885 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:51:30.424524 (+ 8639us) server_negotiation.cc:407] Connection header received
0806 03:51:30.424621 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:51:30.424623 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:51:30.424634 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:51:30.424668 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:51:30.425295 (+ 627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:30.426120 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:30.427127 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:30.427456 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:30.427495 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:30.428883 (+ 1388us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:51:30.428884 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:51:30.428885 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:51:30.428892 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:51:30.430885 (+ 1993us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:51:30.431402 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:51:30.431404 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:51:30.431404 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:51:30.431455 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:51:30.431638 (+ 183us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:51:30.431639 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:51:30.431639 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:51:30.431764 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:51:30.431842 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0806 03:51:30.432030 (+ 188us) server_negotiation.cc:299] Negotiation successful
0806 03:51:30.432104 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260806 03:51:30.443360 11949 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 03:51:30.444195 11949 catalog_manager.cc:1504] Loading token signing keys...
I20260806 03:51:30.451318 11787 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.253.252" port: 35693 } attempt: 3
Aug 06 03:51:30 dist-test-slave-6ntk krb5kdc[11570](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988290, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.253.253@KRBTEST.COM for kudu/127.11.253.252@KRBTEST.COM
I20260806 03:51:30.460498 11789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:30.451571 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.253.252:35693 (local address 127.0.0.1:52362)
0806 03:51:30.451651 (+ 80us) negotiation.cc:107] Waiting for socket to connect
0806 03:51:30.451661 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 03:51:30.451696 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:51:30.452008 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:51:30.452009 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:51:30.452185 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:51:30.452316 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:51:30.452319 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:51:30.453926 (+ 1607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:51:30.453927 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:51:30.454702 (+ 775us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:51:30.454704 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:51:30.455261 (+ 557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:51:30.455262 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:51:30.455329 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:30.455878 (+ 549us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:51:30.455884 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:51:30.457447 (+ 1563us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:51:30.459404 (+ 1957us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:51:30.459406 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:51:30.459406 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:51:30.459609 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:51:30.459840 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:51:30.459841 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:51:30.459841 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:51:30.459904 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:51:30.460189 (+ 285us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:51:30.460190 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:51:30.460254 (+ 64us) client_negotiation.cc:769] Sending connection context
0806 03:51:30.460310 (+ 56us) client_negotiation.cc:240] Negotiation successful
0806 03:51:30.460398 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":62}
I20260806 03:51:30.461028 11899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:30.451780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52362 (local address 127.11.253.252:35693)
0806 03:51:30.451814 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0806 03:51:30.451816 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:51:30.451822 (+ 6us) server_negotiation.cc:407] Connection header received
0806 03:51:30.451841 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:51:30.451842 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:51:30.451850 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:51:30.451889 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:51:30.452453 (+ 564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:30.453170 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:30.454842 (+ 1672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:30.455127 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:30.455179 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:30.457547 (+ 2368us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:51:30.457548 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:51:30.457549 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:51:30.457555 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:51:30.459304 (+ 1749us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:51:30.459692 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:51:30.459693 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:51:30.459693 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:51:30.459768 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:51:30.459973 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:51:30.459974 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:51:30.459974 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:51:30.460100 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:51:30.460153 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0806 03:51:30.460859 (+ 706us) server_negotiation.cc:299] Negotiation successful
0806 03:51:30.460931 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260806 03:51:30.462932 11787 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9f42bb2328ab469ea6b81aa84464da5e: Bootstrap starting.
I20260806 03:51:30.463649 11787 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9f42bb2328ab469ea6b81aa84464da5e: Neither blocks nor log segments found. Creating new log.
I20260806 03:51:30.463918 11787 log.cc:824] T 00000000000000000000000000000000 P 9f42bb2328ab469ea6b81aa84464da5e: Log is configured to *not* fsync() on all Append() calls
I20260806 03:51:30.468766 11787 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9f42bb2328ab469ea6b81aa84464da5e: No bootstrap required, opened a new log
I20260806 03:51:30.471287 11787 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9f42bb2328ab469ea6b81aa84464da5e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbdeb40a549e4c27ba6f21fa8f59c37c" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 35715 } } peers { permanent_uuid: "9f42bb2328ab469ea6b81aa84464da5e" member_type: VOTER last_known_addr { host: "127.11.253.253" port: 39559 } } peers { permanent_uuid: "a3235ea9fa554affbd4f3758620ad411" member_type: VOTER last_known_addr { host: "127.11.253.252" port: 35693 } }
I20260806 03:51:30.471366 11949 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dbdeb40a549e4c27ba6f21fa8f59c37c: Generated new TSK 0
I20260806 03:51:30.471472 11787 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9f42bb2328ab469ea6b81aa84464da5e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f42bb2328ab469ea6b81aa84464da5e, State: Initialized, Role: FOLLOWER
I20260806 03:51:30.471585 11949 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 03:51:30.471568 11787 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9f42bb2328ab469ea6b81aa84464da5e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbdeb40a549e4c27ba6f21fa8f59c37c" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 35715 } } peers { permanent_uuid: "9f42bb2328ab469ea6b81aa84464da5e" member_type: VOTER last_known_addr { host: "127.11.253.253" port: 39559 } } peers { permanent_uuid: "a3235ea9fa554affbd4f3758620ad411" member_type: VOTER last_known_addr { host: "127.11.253.252" port: 35693 } }
I20260806 03:51:30.471974 11787 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9f42bb2328ab469ea6b81aa84464da5e [sys.catalog]: configured and running, proceeding with master startup.
I20260806 03:51:30.472090 11752 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9f42bb2328ab469ea6b81aa84464da5e [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:51:30.472044 11753 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "dbdeb40a549e4c27ba6f21fa8f59c37c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "9f42bb2328ab469ea6b81aa84464da5e"
I20260806 03:51:30.473163 11752 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 9f42bb2328ab469ea6b81aa84464da5e [term 1 FOLLOWER]: Refusing update from remote peer dbdeb40a549e4c27ba6f21fa8f59c37c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 03:51:30.474152 11753 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 9f42bb2328ab469ea6b81aa84464da5e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbdeb40a549e4c27ba6f21fa8f59c37c in term 1.
I20260806 03:51:30.474576 11920 consensus_queue.cc:1035] T 00000000000000000000000000000000 P dbdeb40a549e4c27ba6f21fa8f59c37c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9f42bb2328ab469ea6b81aa84464da5e" member_type: VOTER last_known_addr { host: "127.11.253.253" port: 39559 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260806 03:51:30.482612 11972 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9f42bb2328ab469ea6b81aa84464da5e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 03:51:30.482762 11972 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 03:51:30.483036 11960 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9f42bb2328ab469ea6b81aa84464da5e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dbdeb40a549e4c27ba6f21fa8f59c37c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbdeb40a549e4c27ba6f21fa8f59c37c" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 35715 } } peers { permanent_uuid: "9f42bb2328ab469ea6b81aa84464da5e" member_type: VOTER last_known_addr { host: "127.11.253.253" port: 39559 } } peers { permanent_uuid: "a3235ea9fa554affbd4f3758620ad411" member_type: VOTER last_known_addr { host: "127.11.253.252" port: 35693 } } }
I20260806 03:51:30.483336 11960 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9f42bb2328ab469ea6b81aa84464da5e [sys.catalog]: This master's current role is: FOLLOWER
I20260806 03:51:30.483460 11960 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9f42bb2328ab469ea6b81aa84464da5e [sys.catalog]: SysCatalogTable state changed. Reason: New leader dbdeb40a549e4c27ba6f21fa8f59c37c. Latest consensus state: current_term: 1 leader_uuid: "dbdeb40a549e4c27ba6f21fa8f59c37c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbdeb40a549e4c27ba6f21fa8f59c37c" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 35715 } } peers { permanent_uuid: "9f42bb2328ab469ea6b81aa84464da5e" member_type: VOTER last_known_addr { host: "127.11.253.253" port: 39559 } } peers { permanent_uuid: "a3235ea9fa554affbd4f3758620ad411" member_type: VOTER last_known_addr { host: "127.11.253.252" port: 35693 } } }
I20260806 03:51:30.483589 11960 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9f42bb2328ab469ea6b81aa84464da5e [sys.catalog]: This master's current role is: FOLLOWER
I20260806 03:51:30.483666 11960 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9f42bb2328ab469ea6b81aa84464da5e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "dbdeb40a549e4c27ba6f21fa8f59c37c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbdeb40a549e4c27ba6f21fa8f59c37c" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 35715 } } peers { permanent_uuid: "9f42bb2328ab469ea6b81aa84464da5e" member_type: VOTER last_known_addr { host: "127.11.253.253" port: 39559 } } peers { permanent_uuid: "a3235ea9fa554affbd4f3758620ad411" member_type: VOTER last_known_addr { host: "127.11.253.252" port: 35693 } } }
I20260806 03:51:30.483759 11960 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9f42bb2328ab469ea6b81aa84464da5e [sys.catalog]: This master's current role is: FOLLOWER
I20260806 03:51:30.486639 11974 mvcc.cc:205] Tried to move back new op lower bound from 7315408037765697536 to 7315408037543088128. Current Snapshot: MvccSnapshot[applied={T|T < 7315408037581537280}]
W20260806 03:51:30.503364 11899 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 03:51:30.498210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52364 (local address 127.11.253.252:35693)
0806 03:51:30.498258 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0806 03:51:30.498260 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:51:30.502632 (+ 4372us) server_negotiation.cc:407] Connection header received
0806 03:51:30.502757 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:51:30.502758 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:51:30.502771 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:51:30.502803 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:51:30.503286 (+ 483us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52364: BlockingRecv error: recv got EOF from 127.0.0.1:52364 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
W20260806 03:51:30.504159 11791 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 03:51:30.502255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52094 (local address 127.11.253.253:39559)
0806 03:51:30.502345 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0806 03:51:30.502347 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:51:30.503630 (+ 1283us) server_negotiation.cc:407] Connection header received
0806 03:51:30.503744 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:51:30.503745 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:51:30.503756 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:51:30.503786 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:51:30.504094 (+ 308us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52094: BlockingRecv error: recv got EOF from 127.0.0.1:52094 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
W20260806 03:51:30.504973 11688 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 03:51:30.502301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43090 (local address 127.11.253.254:35715)
0806 03:51:30.502378 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0806 03:51:30.502379 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 03:51:30.504366 (+ 1987us) server_negotiation.cc:407] Connection header received
0806 03:51:30.504467 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:51:30.504469 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:51:30.504482 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:51:30.504512 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:51:30.504886 (+ 374us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43090: BlockingRecv error: recv got EOF from 127.0.0.1:43090 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
W20260806 03:51:30.507344 11791 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 03:51:30.506583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52100 (local address 127.11.253.253:39559)
0806 03:51:30.506616 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0806 03:51:30.506618 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:51:30.506830 (+ 212us) server_negotiation.cc:407] Connection header received
0806 03:51:30.506937 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:51:30.506938 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:51:30.506958 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:51:30.506986 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:51:30.507292 (+ 306us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52100: BlockingRecv error: recv got EOF from 127.0.0.1:52100 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260806 03:51:30.510429 11791 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 03:51:30.509662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52116 (local address 127.11.253.253:39559)
0806 03:51:30.509707 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0806 03:51:30.509709 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:51:30.509939 (+ 230us) server_negotiation.cc:407] Connection header received
0806 03:51:30.510032 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:51:30.510033 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:51:30.510043 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:51:30.510073 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:51:30.510379 (+ 306us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52116: BlockingRecv error: recv got EOF from 127.0.0.1:52116 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260806 03:51:30.514597 11791 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 03:51:30.513611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52122 (local address 127.11.253.253:39559)
0806 03:51:30.513645 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0806 03:51:30.513647 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:51:30.514052 (+ 405us) server_negotiation.cc:407] Connection header received
0806 03:51:30.514152 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:51:30.514153 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:51:30.514164 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:51:30.514193 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:51:30.514546 (+ 353us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52122: BlockingRecv error: recv got EOF from 127.0.0.1:52122 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260806 03:51:30.520821 11791 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 03:51:30.519899 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52126 (local address 127.11.253.253:39559)
0806 03:51:30.519940 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0806 03:51:30.519942 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:51:30.520220 (+ 278us) server_negotiation.cc:407] Connection header received
0806 03:51:30.520330 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:51:30.520332 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:51:30.520343 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:51:30.520374 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:51:30.520741 (+ 367us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52126: BlockingRecv error: recv got EOF from 127.0.0.1:52126 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260806 03:51:30.531270 11791 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 03:51:30.530329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52138 (local address 127.11.253.253:39559)
0806 03:51:30.530376 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0806 03:51:30.530378 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:51:30.530639 (+ 261us) server_negotiation.cc:407] Connection header received
0806 03:51:30.530747 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:51:30.530748 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:51:30.530761 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:51:30.530805 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:51:30.531192 (+ 387us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52138: BlockingRecv error: recv got EOF from 127.0.0.1:52138 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
W20260806 03:51:30.549630 11791 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 03:51:30.548702 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52154 (local address 127.11.253.253:39559)
0806 03:51:30.548748 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0806 03:51:30.548750 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:51:30.549024 (+ 274us) server_negotiation.cc:407] Connection header received
0806 03:51:30.549128 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:51:30.549130 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:51:30.549142 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:51:30.549173 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:51:30.549565 (+ 392us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52154: BlockingRecv error: recv got EOF from 127.0.0.1:52154 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260806 03:51:30.584200 11791 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 03:51:30.583173 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52166 (local address 127.11.253.253:39559)
0806 03:51:30.583225 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0806 03:51:30.583227 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:51:30.583515 (+ 288us) server_negotiation.cc:407] Connection header received
0806 03:51:30.583631 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:51:30.583632 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:51:30.583646 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:51:30.583677 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:51:30.584121 (+ 444us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52166: BlockingRecv error: recv got EOF from 127.0.0.1:52166 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
W20260806 03:51:30.650640 11791 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 03:51:30.649743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52176 (local address 127.11.253.253:39559)
0806 03:51:30.649835 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0806 03:51:30.649837 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:51:30.649945 (+ 108us) server_negotiation.cc:407] Connection header received
0806 03:51:30.649982 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:51:30.649983 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:51:30.649994 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:51:30.650040 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:51:30.650568 (+ 528us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52176: BlockingRecv error: recv got EOF from 127.0.0.1:52176 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
W20260806 03:51:30.780679 11791 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 03:51:30.779918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52188 (local address 127.11.253.253:39559)
0806 03:51:30.780008 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0806 03:51:30.780010 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:51:30.780058 (+ 48us) server_negotiation.cc:407] Connection header received
0806 03:51:30.780125 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:51:30.780126 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:51:30.780137 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:51:30.780181 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:51:30.780600 (+ 419us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52188: BlockingRecv error: recv got EOF from 127.0.0.1:52188 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
W20260806 03:51:31.039068 11791 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 03:51:31.038306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52198 (local address 127.11.253.253:39559)
0806 03:51:31.038391 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0806 03:51:31.038393 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:51:31.038404 (+ 11us) server_negotiation.cc:407] Connection header received
0806 03:51:31.038433 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:51:31.038434 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:51:31.038445 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:51:31.038505 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:51:31.038988 (+ 483us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52198: BlockingRecv error: recv got EOF from 127.0.0.1:52198 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
I20260806 03:51:31.076074 12062 sys_catalog.cc:455] T 00000000000000000000000000000000 P dbdeb40a549e4c27ba6f21fa8f59c37c [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "dbdeb40a549e4c27ba6f21fa8f59c37c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbdeb40a549e4c27ba6f21fa8f59c37c" member_type: VOTER last_known_addr { host: "127.11.253.254" port: 35715 } } peers { permanent_uuid: "9f42bb2328ab469ea6b81aa84464da5e" member_type: VOTER last_known_addr { host: "127.11.253.253" port: 39559 } } peers { permanent_uuid: "a3235ea9fa554affbd4f3758620ad411" member_type: VOTER last_known_addr { host: "127.11.253.252" port: 35693 } } }
I20260806 03:51:31.076254 12062 sys_catalog.cc:458] T 00000000000000000000000000000000 P dbdeb40a549e4c27ba6f21fa8f59c37c [sys.catalog]: This master's current role is: LEADER
I20260806 03:51:31.410732 11918 catalog_manager.cc:1259] Loaded cluster ID: 42700d7593994432bca26b908debdcfc
I20260806 03:51:31.410815 11918 catalog_manager.cc:1552] T 00000000000000000000000000000000 P a3235ea9fa554affbd4f3758620ad411: loading cluster ID for follower catalog manager: success
I20260806 03:51:31.411598 11918 catalog_manager.cc:1574] T 00000000000000000000000000000000 P a3235ea9fa554affbd4f3758620ad411: acquiring CA information for follower catalog manager: success
I20260806 03:51:31.412029 11918 catalog_manager.cc:1602] T 00000000000000000000000000000000 P a3235ea9fa554affbd4f3758620ad411: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 03:51:31.483341 11972 catalog_manager.cc:1259] Loaded cluster ID: 42700d7593994432bca26b908debdcfc
I20260806 03:51:31.483428 11972 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 9f42bb2328ab469ea6b81aa84464da5e: loading cluster ID for follower catalog manager: success
I20260806 03:51:31.484457 11972 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 9f42bb2328ab469ea6b81aa84464da5e: acquiring CA information for follower catalog manager: success
I20260806 03:51:31.484903 11972 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 9f42bb2328ab469ea6b81aa84464da5e: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 03:51:31.555079 12074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:31.552426 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41994 (local address 127.11.253.253:39559)
0806 03:51:31.552584 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0806 03:51:31.552586 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:51:31.552612 (+ 26us) server_negotiation.cc:407] Connection header received
0806 03:51:31.552638 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:51:31.552639 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:51:31.552697 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:51:31.552766 (+ 69us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 03:51:31.552941 (+ 175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:31.553452 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:31.554306 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:31.554524 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:31.554581 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:31.554733 (+ 152us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 03:51:31.554850 (+ 117us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 03:51:31.554911 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0806 03:51:31.554989 (+ 78us) server_negotiation.cc:299] Negotiation successful
0806 03:51:31.555007 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":89,"threads_started":1}
I20260806 03:51:31.559471 12085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:31.557025 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51288 (local address 127.11.253.252:35693)
0806 03:51:31.557155 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0806 03:51:31.557157 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:51:31.557165 (+ 8us) server_negotiation.cc:407] Connection header received
0806 03:51:31.557205 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:51:31.557205 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:51:31.557245 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:51:31.557315 (+ 70us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 03:51:31.557526 (+ 211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:31.558083 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:31.558799 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:31.558997 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:31.559048 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:31.559170 (+ 122us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 03:51:31.559273 (+ 103us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 03:51:31.559326 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0806 03:51:31.559386 (+ 60us) server_negotiation.cc:299] Negotiation successful
0806 03:51:31.559408 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":68,"threads_started":1}
I20260806 03:51:31.564345 12074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:31.561281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41996 (local address 127.11.253.253:39559)
0806 03:51:31.561439 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0806 03:51:31.561441 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:51:31.561787 (+ 346us) server_negotiation.cc:407] Connection header received
0806 03:51:31.561884 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:51:31.561885 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:51:31.561898 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:51:31.561950 (+ 52us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 03:51:31.562162 (+ 212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:31.562854 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:31.563615 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:31.563839 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:31.563921 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:31.564090 (+ 169us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 03:51:31.564129 (+ 39us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 03:51:31.564180 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0806 03:51:31.564238 (+ 58us) server_negotiation.cc:299] Negotiation successful
0806 03:51:31.564260 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130}
I20260806 03:51:31.564543 12085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:31.561475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51294 (local address 127.11.253.252:35693)
0806 03:51:31.561515 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0806 03:51:31.561517 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:51:31.561626 (+ 109us) server_negotiation.cc:407] Connection header received
0806 03:51:31.561703 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:51:31.561705 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:51:31.561717 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:51:31.561774 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 03:51:31.561949 (+ 175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:31.562659 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:31.563654 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:31.563950 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:31.564013 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:31.564171 (+ 158us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 03:51:31.564204 (+ 33us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 03:51:31.564417 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0806 03:51:31.564438 (+ 21us) server_negotiation.cc:299] Negotiation successful
0806 03:51:31.564462 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260806 03:51:31.564764 12092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:31.561354 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39880 (local address 127.11.253.254:35715)
0806 03:51:31.561550 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0806 03:51:31.561552 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:51:31.561564 (+ 12us) server_negotiation.cc:407] Connection header received
0806 03:51:31.561656 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:51:31.561657 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:51:31.561714 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:51:31.561808 (+ 94us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 03:51:31.561986 (+ 178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:31.562479 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:31.563576 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:31.563885 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:31.563948 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:31.564175 (+ 227us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 03:51:31.564288 (+ 113us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 03:51:31.564436 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0806 03:51:31.564640 (+ 204us) server_negotiation.cc:299] Negotiation successful
0806 03:51:31.564666 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":173,"thread_start_us":65,"threads_started":1}
I20260806 03:51:31.566664 12279 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskToVWRe/build/release/bin/kudu with pid 11592
I20260806 03:51:31.577805 12279 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskToVWRe/build/release/bin/kudu with pid 11693
I20260806 03:51:31.589696 12279 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskToVWRe/build/release/bin/kudu with pid 11796
2026-08-06T03:51:31Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (3043 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (6132 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (286959 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST